One Vote or Ten: token-mean vs sample-mean

Does a 5000-token answer outweigh a 500-token one? Sample-mean gives each response a vote, token-mean each token, and switching has a hidden cost.

August 30, 2026 · 9 min · 1843 words · njx

“Wrong? Write Longer”: What GRPO Normalizers Skew

Part of R1-Zero’s growing length may be the optimizer’s own bias: 1/|y| favors long wrong answers and dividing by std overweights extreme questions.

August 29, 2026 · 10 min · 2061 words · njx