The Ratio's Wrong Unit: GSPO Goes Sequence-Level
Qwen says GRPO’s token-level importance ratio misuses importance sampling and breaks MoE training. GSPO lifts ratio, clipping and updates to sequence level.
Qwen says GRPO’s token-level importance ratio misuses importance sampling and breaks MoE training. GSPO lifts ratio, clipping and updates to sequence level.