[RuBing-Yang]

This pull request introduces a unified, two-stage quantization pipeline for the Hy4 model, enabling both vLLM activation calibration and mixed-precision weight quantization using a single YAML configuration. The changes include a new one-click shell script, detailed documentation, and robust support for MTP (Multi-Token Prediction) calibration, along with key refactors and improvements to the calibration Python script. The most important changes are summarized below:

Pipeline & Configuration Unification:

• Added a comprehensive YAML template (Hy4_template.yaml) that drives both the vLLM activation calibration and the mixed-precision NVFP4/FP8 quantization stages, ensuring consistent configuration and path management across both steps.

• Introduced the run_hy4_nvfp4_quant.sh script, which orchestrates both pipeline stages, handles CLI overrides, environment setup, and logging, and allows users to skip individual stages as needed.

Documentation:

• Substantially expanded the scripts/ptq/README.md with a new section detailing the Hy4 pipeline, including prerequisites (vLLM patching, Ray cluster setup), environment variables, YAML/CLI integration, and troubleshooting for MTP calibration.

MTP Calibration Improvements:

• Added the --mtp-calib-max-tokens argument to tools/run_vllm_calibrate.py and the YAML template, allowing users to specify a generation length for proper MTP calibration. This ensures the draft model is exercised and valid MTP statistics are collected for NVFP4 quantization. [1] [2] [3]

• Enhanced logic in run_vllm_calibrate.py to warn users if MTP calibration would be ineffective (i.e., max_tokens=1), and to use the correct token count for MTP runs.

Refactoring & Robustness:

• Refactored tools/run_vllm_calibrate.py to import worker helpers (for activation hooks, stats printing, etc.) in a way that is robust to both patched and unpatched vLLM installations, ensuring compatibility in distributed Ray environments. [1] [2]

• Replaced lambda-based Ray worker invocations with explicit, importable functions (using functools.partial), improving serialization safety and maintainability. [1] [2] [3]

These changes collectively provide a robust, user-friendly, and maintainable workflow for Hy4 quantization, with special attention to MTP calibration correctness and multi-node distributed execution.

References: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]