Skip to content

added vllm_disagg profiling - #190

Open
AakarshAMD wants to merge 1 commit into
ROCm:developfrom
AakarshAMD:vllm_profiling_addition
Open

added vllm_disagg profiling#190
AakarshAMD wants to merge 1 commit into
ROCm:developfrom
AakarshAMD:vllm_profiling_addition

Conversation

@AakarshAMD

Copy link
Copy Markdown

Motivation

Add an opt-in profiling workflow for vLLM disaggregated serving using rocprofv3, ROCtx, and MoRIIO instrumentation while preserving the normal non-profiling workflow.

Technical Details

  • Added a dedicated ROCm 7.2.3 profiling Dockerfile with rocprofv3 and MoRIIO request-ID instrumentation.
  • Added benchmark_xPyD_profile.sh for profiling-specific request IDs, timing manifests, warmup control, and bounded workloads.
  • Added conditional kernel/marker capture and graceful worker finalization.
  • Added composite request-ID correlation, combined traces, request statistics, and kernel categorization.
  • Added true local-rank-0 selection and cross-node clock alignment for combined traces.
  • Added validation for capture completeness, malformed durations, request identity, MoRI transfer attribution, and analysis failures.
  • Profiling requires RUN_PROFILE=1 with CONNECTOR=moriio; normal runs remain unchanged.

Test Plan

  • Build the dedicated profiling image using the profiling target.
  • Run profiled 1P1D and 2P2D configurations across TP and EP models.
  • Verify benchmark completion, capture finalization, request-ID consistency, ROCtx markers, MoRI attribution, combined traces, and kernel-analysis outputs.

Test Result

All validation jobs below passed on their first submitted attempt:

  • Llama 3.3 70B FP8-KV — 1P1D, TP8 (jobs 209995 and 210278)
  • GPT-OSS 120B — 1P1D, TP8 (job 210279)
  • DeepSeek-R1 — 1P1D, EP8 (job 210280)
  • DeepSeek-V3 — 1P1D, EP8 (job 210281)
  • DeepSeek-R1 — 2P2D, EP16/stage (job 210282)
  • DeepSeek-V3 — 2P2D, EP16/stage (job 210283)

Submission Checklist

@AakarshAMD
AakarshAMD force-pushed the vllm_profiling_addition branch from ca0cdff to 9921627 Compare August 6, 2026 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant