“Wrong? Write Longer”: What GRPO Normalizers Skew

Part of R1-Zero’s growing length may be the optimizer’s own bias: 1/|y| favors long wrong answers and dividing by std overweights extreme questions.

August 29, 2026 · 10 min · 2061 words · njx