One Model, Two Probabilities: Cost of Mismatch

Rollout and training engines compute different probabilities for one model, so on-policy RL quietly turns off-policy. Where it comes from and how to fix it.

August 27, 2026 · 12 min · 2377 words · njx