“Wrong? Write Longer”: What GRPO Normalizers Skew
Part of R1-Zero’s growing length may be the optimizer’s own bias: 1/|y| favors long wrong answers and dividing by std overweights extreme questions.
Part of R1-Zero’s growing length may be the optimizer’s own bias: 1/|y| favors long wrong answers and dividing by std overweights extreme questions.