Skip to content

docs: document the internal APIs that the OpenTelemetry agent instruments - #3473

Merged
pjfanning merged 1 commit into
apache:mainfrom
pjfanning:docs/otel-instrumentation-points
Aug 28, 2026
Merged

docs: document the internal APIs that the OpenTelemetry agent instruments#3473
pjfanning merged 1 commit into
apache:mainfrom
pjfanning:docs/otel-instrumentation-points

Conversation

@pjfanning

Copy link
Copy Markdown
Member

Motivation

The OpenTelemetry Java agent propagates trace context through pekko actors, streams and remoting by attaching bytecode advice to specific internal methods. Its muzzle checks verify the classes and methods the advice code calls, not the method matchers, so a matched method that is renamed, inlined or restructured just stops matching — no error, no log, silently broken propagation.

This is the pekko-core counterpart of apache/pekko-http#1240, which does the same for pekko-http.

Modification

  • Documented each matched method where it is declared: which agent depends on it, what the agent uses it for, and a link to the instrumentation source.
  • Marked those methods @noinline so the compiler cannot inline them out of the bytecode, following the existing precedent in Envelope.copy, MessageBuffer and Dispatch.stop.
  • Listed the RemoteInstrument.identifier values known to be taken (0 Lightbend Telemetry, 1 Pekko's own LoggingRemoteInstrument, 8 Kamon, 9 OpenTelemetry) so the next implementer does not have to grep other projects for a free one.

No API changes, no @InternalStableApi added.

Documented symbols:

Module Symbols
pekko-actor Dispatcher.dispatch, ActorCell.invoke, ActorCell.systemInvoke, DefaultSystemMessageQueue.systemEnqueue, LightArrayRevolverScheduler.schedule/scheduleOnce
pekko-stream GraphInterpreter.processPush
pekko-remote, artery RemoteInstruments.create/serialize/deserialize, ReusableOutboundEnvelope.init/copy/clear, ReusableInboundEnvelope.init/clear, artery.MessageDispatcher.dispatch
pekko-remote, classic EndpointManager.Send, EndpointWriter.writeSend, DefaultMessageDispatcher.dispatch, PekkoPduProtobufCodec.constructMessage/decodeMessage

Links pin the pekko-actor and pekko-http instrumentation at OpenTelemetry commit 6f9ca5672ce84edbbe36ce0e14386c31d68f479f (the same commit referenced from apache/pekko-http#1240). The remoting instrumentation is not merged upstream yet, so those comments link to open-telemetry/opentelemetry-java-instrumentation#19823 instead.

Result

The load-bearing internal APIs are recorded next to the code, so a refactor of one of them is visible to the person making it and can be called out in release notes. Three new InstrumentationPointsSpec suites assert by reflection that each matched name, arity and parameter type is still present in the bytecode, so a change that would silently disable the agent fails the build instead.

Tests

  • sbt "actor-tests/testOnly org.apache.pekko.dispatch.InstrumentationPointsSpec" - 6 passed
  • sbt "stream-tests/testOnly org.apache.pekko.stream.impl.fusing.InstrumentationPointsSpec" - 1 passed
  • sbt "remote/testOnly org.apache.pekko.remote.InstrumentationPointsSpec" - 10 passed
  • sbt actor/mimaReportBinaryIssues stream/mimaReportBinaryIssues remote/mimaReportBinaryIssues - clean
  • sbt scalafmtCheckAll scalafmtSbtCheck - pass
  • sbt headerCreateAll - headers added for the three new files

References

Fixes #3472
Refs apache/pekko-http#1240
Refs open-telemetry/opentelemetry-java-instrumentation#19823

…ents

Motivation:
The OpenTelemetry Java agent propagates trace context through pekko actors,
streams and remoting by attaching bytecode advice to specific internal methods.
Its muzzle checks verify the classes the advice code calls, not the method
matchers, so renaming, inlining or restructuring a matched method silently stops
context propagation with no error and no log.

Modification:
Document the matched methods where they are declared, naming the agent, saying
what it uses the method for and linking to the instrumentation source. Mark them
`@noinline` so the compiler cannot inline them out of the bytecode. List the
RemoteInstrument identifiers known to be taken by other projects so that the next
implementer can pick a free one. No API changes.

Result:
The load-bearing internal APIs are recorded next to the code, and new
InstrumentationPointsSpec suites fail loudly if a matched name, arity or
parameter type disappears from the bytecode.

Tests:
- sbt "actor-tests/testOnly org.apache.pekko.dispatch.InstrumentationPointsSpec" - 6 passed
- sbt "stream-tests/testOnly org.apache.pekko.stream.impl.fusing.InstrumentationPointsSpec" - 1 passed
- sbt "remote/testOnly org.apache.pekko.remote.InstrumentationPointsSpec" - 10 passed
- sbt actor/mimaReportBinaryIssues stream/mimaReportBinaryIssues remote/mimaReportBinaryIssues - clean
- sbt scalafmtCheckAll scalafmtSbtCheck - pass
- sbt headerCreateAll - headers added for the three new files

References:
Fixes apache#3472, Refs apache/pekko-http#1240,
Refs open-telemetry/opentelemetry-java-instrumentation#19823
@pjfanning
pjfanning merged commit fde63d4 into apache:main Aug 28, 2026
10 checks passed
@pjfanning
pjfanning deleted the docs/otel-instrumentation-points branch August 28, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document the internal APIs that the OpenTelemetry agent instruments

2 participants