Support: trace native decode overlap

[Crane-Liu]

Summary

• emit correlated host-clock spans for graph build, frame staging, native prepare, launch acceptance, device execution, ordered release, and post-fence retirement
• carry run_id / dispatch_id / slot_id / generation through the trace and add --assert-native-overlap for machine-verifiable ordering
• make the existing A2/A3 concurrent-prepare stress test prove 39 adjacent overlap pairs without changing scheduling behavior

Testing

• pre-commit run --files ...
• pytest -q tests/ut/py/test_strace_timing.py (18 passed)
• _task_interface build
• test_orchestrator (54 passed)
• test_scheduler (66 passed)
• test_chip_run_lane (17 passed)
• A2/A3 architecture precheck via task-submit
• build_package_a2a3 with PTO-ISA pin 0cefc9a5a1c24c62655cc345d408559595a8af32
• A2/A3 onboard test_concurrent_prepare_overlap via task-submit (1 passed, 40 runs / 39 adjacent pairs)
• sim tests not run (not required for this A2/A3 trace PR)

[coderabbitai]

📝 Walkthrough

Walkthrough

The change adds host-clock tracing for native pipeline lifecycle phases. It records dispatch identities and validates adjacent-dispatch preparation overlap, launch ordering, and ordered release through strace_timing.py, with unit, stress, and documentation updates.

Changes

Native pipeline tracing and validation

Layer / File(s)	Summary
Tracing contracts and metadata src/common/hierarchical/pipeline_trace.h, src/common/log/include/common/strace.h, src/common/hierarchical/types.h, src/common/worker/native_run_context.h	Defines pipeline span emitters, conditional tracing macros, run timestamps, and native dispatch identity accessors.
Runtime pipeline span instrumentation src/common/hierarchical/orchestrator.cpp, src/common/hierarchical/worker_manager.cpp, src/common/platform/onboard/host/c_api_shared.cpp	Emits graph-build, frame-staging, native preparation, device execution, launch-acceptance, ordered-release, and post-fence-retirement spans.
Overlap validation and verification coverage simpler_setup/tools/strace_timing.py, tests/ut/py/test_strace_timing.py, tests/st/a2a3/host_build_graph/concurrent_prepare_stress/test_concurrent_prepare_stress.py, docs/dfx/host-trace.md	Parses and validates pipeline records, adds --assert-native-overlap, and covers valid overlap, invalid ordering, CLI failures, stress output, and trace usage.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Orchestrator
  participant WorkerThread_submit_dispatch
  participant NativeRunContext
  participant strace_timing
  Orchestrator->>Orchestrator: Emit graph_build and retirement spans
  WorkerThread_submit_dispatch->>WorkerThread_submit_dispatch: Emit frame_staging span
  NativeRunContext->>NativeRunContext: Record device trace start
  NativeRunContext->>strace_timing: Provide native pipeline spans
  strace_timing->>strace_timing: Validate adjacent dispatch overlap and release order

Loading

Possibly related PRs

• hw-native-sys/simpler#1650: Adds the concurrent native-preparation pipeline that this PR traces and validates.
• hw-native-sys/simpler#1654: Refactors native pipeline preparation and execution paths instrumented by this PR.
• hw-native-sys/simpler#1694: Introduces related native pipeline tracing and lifecycle changes.

Poem

Rabbit hops through spans of light,
Dispatch trails align just right.
Prep overlaps device flight,
Releases come in ordered sight.
Tests catch errors, crisp and bright!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 10.26% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the main change: adding tracing support for native decode overlap.
Description check	✅ Passed	The description accurately explains the tracing, validation, stress-test, and testing changes in the pull request.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/dfx/host-trace.md`:
- Around line 122-125: Update the adjacent-dispatch overlap requirements in the
command documentation to state that native_prepare(N+1) must finish after
device_execution(N) starts and no later than device_execution(N) ends, while
preserving the existing start-before-end and ordered_release constraints.

In `@src/common/hierarchical/pipeline_trace.h`:
- Around line 49-53: Update the STRACE formatting call in the pipeline trace
emission to populate the tid field with the actual native thread ID, matching
the mechanism used by strace_tid(), while retaining getpid() for pid and leaving
the remaining fields unchanged.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 68aad3ec-83ea-4ed3-9b56-9b546304d1ca

📥 Commits

Reviewing files that changed from the base of the PR and between e4ab544 and 32ed069.

📒 Files selected for processing (11)

• docs/dfx/host-trace.md
• simpler_setup/tools/strace_timing.py
• src/common/hierarchical/orchestrator.cpp
• src/common/hierarchical/pipeline_trace.h
• src/common/hierarchical/types.h
• src/common/hierarchical/worker_manager.cpp
• src/common/log/include/common/strace.h
• src/common/platform/onboard/host/c_api_shared.cpp
• src/common/worker/native_run_context.h
• tests/st/a2a3/host_build_graph/concurrent_prepare_stress/test_concurrent_prepare_stress.py
• tests/ut/py/test_strace_timing.py