Repository navigation

[feat] Support value head and critic loss - #2188

Open

YanhuiDua wants to merge 10 commits into

stack/rl-data-packingfrom

stack/rl-value-head

Open

[feat] Support value head and critic loss#2188
YanhuiDua wants to merge 10 commits into
stack/rl-data-packingfrom
stack/rl-value-head

Commits on Oct 26, 2026
- support value head
  PengchengShi00
  committed
  27eb554
- Rename repo root to top-level HF key in tests
  PengchengShi00
  committed
  41044fb
- fix tests
  PengchengShi00
  committed
  5e9818f
- [Fix] Preserve remapped critic value heads
  PengchengShi00
  committed
  28f3e49
- [Fix] Separate FP32 LM and value head policies
  PengchengShi00
  committed
  7a7f639
- refactor init value head
  PengchengShi00
  committed
  0877871
- refactor: move critic_loss from xtuner.v1.rl.loss to xtuner.v1.loss
  PengchengShi00
  committed
  ce9c228
- [Refactor] Centralize tie_head and head_fp32 policies for value head
  PengchengShi00
  committed
  da2e7a5
- [Refactor] Use tie_head and head_fp32 across dense and MoE models
  PengchengShi00
  committed
  27089af
- fix loss compute on cpu
  PengchengShi00
  committed
  1a06068