<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>normalization on Njx&#39;Log</title>
    <link>https://njx-njx.github.io/en/tags/normalization/</link>
    <description>Recent content in normalization on Njx&#39;Log</description>
    <image>
      <title>Njx&#39;Log</title>
      <url>https://njx-njx.github.io/blog-cover.jpg</url>
      <link>https://njx-njx.github.io/blog-cover.jpg</link>
    </image>
    <generator>Hugo</generator>
    <language>en</language>
    <lastBuildDate>Tue, 27 Oct 2026 15:14:17 +0800</lastBuildDate>
    <atom:link href="https://njx-njx.github.io/en/tags/normalization/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>“Wrong? Write Longer”: What GRPO Normalizers Skew</title>
      <link>https://njx-njx.github.io/en/posts/dr-grpo-length-and-difficulty-bias/</link>
      <pubDate>Sat, 29 Aug 2026 12:05:00 +0800</pubDate>
      <guid>https://njx-njx.github.io/en/posts/dr-grpo-length-and-difficulty-bias/</guid>
      <description>Part of R1-Zero&amp;rsquo;s growing length may be the optimizer&amp;rsquo;s own bias: 1/|y| favors long wrong answers and dividing by std overweights extreme questions.</description>
      <content:encoded><![CDATA[<p>R1-Zero&rsquo;s “length and reward rising together” has been treated as the signature curve of emergent reasoning, but Sea AI Lab&rsquo;s Dr.GRPO paper points out that half of that rise may be the optimizer&rsquo;s own preference: 1/|y| makes the model lean toward writing longer when it answers wrong, and dividing by std lets questions at the difficulty extremes take more of the weight. A mechanistic breakdown of two normalization terms and two biases, plus an implementation-level bias hidden in masked_mean.</p>
<blockquote>
<p>This post has a companion <a href="https://njx-njx.github.io/reports/dr-grpo-length-and-difficulty-bias/">interactive report</a>
 (in Chinese), with drill-down worked examples of the biases, a side-by-side comparison of masked_mean implementations, and a decision matrix for choosing a fix.</p>
</blockquote>
<h2 id="an-over-trusted-curve">An Over-Trusted Curve</h2>
<p>The most famous plot from R1-Zero training is the one where response length and training reward rise in lockstep. The community was quick to spin a story around it: the model has learned to think longer, and long CoT keeps taking shape through RL.</p>
<p>Sea AI Lab&rsquo;s Dr.GRPO paper (arXiv:2503.20783) poses a simpler question: on the plateau where reward has stopped rising, why does length keep growing? After pinning down the causes one by one, the paper concludes that this is not the model learning to reason more; rather, <strong>the optimizer keeps pushing the model to generate longer responses</strong>. The driving force comes from two normalization terms in the GRPO objective: they adjust the model&rsquo;s optimization direction in opposite ways for correct and incorrect responses, and their effect on incorrect responses points to one and the same outcome: <strong>when the answer is wrong, the longer the response, the less it gets penalized.</strong></p>
<h2 id="bias-1-1y--per-token-averaging-dilutes-the-penalty">Bias 1: 1/|y| — Per-Token Averaging Dilutes the Penalty</h2>
<p>The GRPO objective (Equation 3 in the paper) contains an inconspicuous denominator:</p>
<span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mi mathvariant="script">J</mi><mrow><mi>G</mi><mi>R</mi><mi>P</mi><mi>O</mi></mrow></msub><mo>=</mo><mi mathvariant="double-struck">E</mi><mrow><mo fence="true">[</mo><mfrac><mn>1</mn><mi>G</mi></mfrac><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>G</mi></munderover><mfrac><mn>1</mn><mrow><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow></mfrac><munderover><mo>∑</mo><mrow><mi>t</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow></munderover><mi>min</mi><mo>⁡</mo><mrow><mo fence="true">(</mo><msub><mi>w</mi><mrow><mi>i</mi><mo separator="true">,</mo><mi>t</mi></mrow></msub><msub><mover accent="true"><mi>A</mi><mo>^</mo></mover><mi>i</mi></msub><mo separator="true">,</mo><mtext> </mtext><mrow><mi mathvariant="normal">c</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">i</mi><mi mathvariant="normal">p</mi></mrow><mo stretchy="false">(</mo><msub><mi>w</mi><mrow><mi>i</mi><mo separator="true">,</mo><mi>t</mi></mrow></msub><mo separator="true">,</mo><mn>1</mn><mo>−</mo><mi>ε</mi><mo separator="true">,</mo><mn>1</mn><mo>+</mo><mi>ε</mi><mo stretchy="false">)</mo><msub><mover accent="true"><mi>A</mi><mo>^</mo></mover><mi>i</mi></msub><mo fence="true">)</mo></mrow><mo fence="true">]</mo></mrow></mrow><annotation encoding="application/x-tex">\mathcal{J}_{GRPO} = \mathbb{E}\left[\frac{1}{G}\sum_{i=1}^{G} \frac{1}{|y_i|}\sum_{t=1}^{|y_i|}\min\left(w_{i,t}\hat{A}_i,\ \mathrm{clip}(w_{i,t},1-\varepsilon,1+\varepsilon)\hat{A}_i\right)\right]</annotation></semantics></math></span><p>What <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>1</mn><mi mathvariant="normal">/</mi><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">1/|y_i|</annotation></semantics></math></span> means is this: within a single response, first average the gradient terms over all of its tokens, then add up the <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi></mrow><annotation encoding="application/x-tex">G</annotation></semantics></math></span> responses. In other words, <strong>the gradient weight each token actually receives is <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><msub><mover accent="true"><mi>A</mi><mo>^</mo></mover><mi>i</mi></msub><mi mathvariant="normal">/</mi><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">\hat{A}_i/|y_i|</annotation></semantics></math></span></strong>. A concrete worked example: sample two responses to the same question, one 100 tokens long and one 1000 tokens long, both with an advantage of absolute value 1 (opposite in sign):</p>
<ul>
<li>Correct (<span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>A</mi><mo>^</mo></mover><mo>=</mo><mo>+</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">\hat{A}=+1</annotation></semantics></math></span>): each token of the short response is reinforced by <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>1</mn><mi mathvariant="normal">/</mi><mn>100</mn><mo>=</mo><mn>0.01</mn></mrow><annotation encoding="application/x-tex">1/100 = 0.01</annotation></semantics></math></span>, while each token of the long response is reinforced by <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>1</mn><mi mathvariant="normal">/</mi><mn>1000</mn><mo>=</mo><mn>0.001</mn></mrow><annotation encoding="application/x-tex">1/1000 = 0.001</annotation></semantics></math></span>. <strong>Among correct responses, the shorter, the more favored</strong>;</li>
<li>Incorrect (<span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>A</mi><mo>^</mo></mover><mo>=</mo><mo>−</mo><mn>1</mn></mrow><annotation encoding="application/x-tex">\hat{A}=-1</annotation></semantics></math></span>): each token of the short response is penalized by 0.01, while each token of the long response is penalized by only 0.001. <strong>Among incorrect responses, the longer, the less penalized</strong> — a tenfold difference.</li>
</ul>
<p>This is not an anomaly caused by a math error; it is something the objective writes into every update step: correct answers must be concise; wrong answers may as well be long-winded. Late in training, once reward plateaus, the model&rsquo;s room for improvement on the first half is used up, while the second half keeps pushing the model to generate longer responses — incorrect responses in particular keep getting longer. Fig.5 Plot 2 in the paper&rsquo;s Figure 4 clearly shows this phase: GRPO&rsquo;s length curve keeps rising after reward plateaus.</p>
<h2 id="bias-2-dividing-by-std--extreme-difficulty-questions-take-more-weight">Bias 2: Dividing by std — Extreme-Difficulty Questions Take More Weight</h2>
<p>The second normalization term sits in the advantage:</p>
<span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mover accent="true"><mi>A</mi><mo>^</mo></mover><mi>i</mi></msub><mo>=</mo><mfrac><mrow><msub><mi>R</mi><mi>i</mi></msub><mo>−</mo><mrow><mi mathvariant="normal">m</mi><mi mathvariant="normal">e</mi><mi mathvariant="normal">a</mi><mi mathvariant="normal">n</mi></mrow><mo stretchy="false">(</mo><mo stretchy="false">{</mo><msub><mi>R</mi><mi>j</mi></msub><msubsup><mo stretchy="false">}</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mi>G</mi></msubsup><mo stretchy="false">)</mo></mrow><mrow><mrow><mi mathvariant="normal">s</mi><mi mathvariant="normal">t</mi><mi mathvariant="normal">d</mi></mrow><mo stretchy="false">(</mo><mo stretchy="false">{</mo><msub><mi>R</mi><mi>j</mi></msub><msubsup><mo stretchy="false">}</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mi>G</mi></msubsup><mo stretchy="false">)</mo></mrow></mfrac></mrow><annotation encoding="application/x-tex">\hat{A}_i = \frac{R_i - \mathrm{mean}(\{R_j\}_{j=1}^G)}{\mathrm{std}(\{R_j\}_{j=1}^G)}</annotation></semantics></math></span><p>Dividing by the within-group reward standard deviation is meant to put advantages on a common scale (a standard RL trick). But the paper points out a key difference: classic RL advantage normalization computes the std over <strong>the whole batch</strong>, with all samples sharing one divisor; GRPO <strong>computes the std separately for each question</strong>, so every question has its own divisor — and different divisors mean different weights.</p>
<p>One more worked example: for question 1, all 8 sampled responses are correct (std≈0.35, <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>A</mi><mo>^</mo></mover></mrow><annotation encoding="application/x-tex">\hat{A}</annotation></semantics></math></span>≈+2.9 on the z-score scale); for question 2, the 8 samples are four correct and four wrong (std≈0.52, <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mover accent="true"><mi>A</mi><mo>^</mo></mover></mrow><annotation encoding="application/x-tex">\hat{A}</annotation></semantics></math></span>≈±1.9). The more extreme the question — the all-correct giveaway, the all-wrong dead end — the smaller its std, and the larger its weight after the division. The model spends the most update budget on questions it already gets entirely right (the easy ones) and on questions it cannot solve at all (never answered correctly even once), while questions that are “partly right, partly wrong, right in the zone where learning pays off” get the least weight. This is what the paper calls <strong>difficulty bias</strong>.</p>
<p>A caveat on scope: std normalization is not all downside. Its design intent, amplifying the signal in low-variance cases, is to let the model learn something even from questions it gets entirely right. What the paper criticizes is the unequal weighting caused by <strong>per-question normalization</strong>, not normalization itself.</p>
<h2 id="bias-3-hidden-in-the-implementation-not-the-formula">Bias 3: Hidden in the Implementation, Not the Formula</h2>
<p>The third finding may hit engineers harder. The PPO objective (Equation 2 in the paper) <strong>sums</strong> over the response&rsquo;s tokens, with no step that divides by length. But the paper examined mainstream open-source implementations and found that the losses in frameworks such as trl and verl all contain a <code>masked_mean</code> line: it averages using the response&rsquo;s actual token count as the divisor (<code>loss.sum(dim) / mask.sum(dim)</code>). In other words, <strong>the formula has no length bias, but the implementation adds one</strong>.</p>
<p>How did this happen? The paper&rsquo;s conjecture traces it to historical practice: in pretraining, all data is packed into fixed-length contexts, and <code>loss.mean(-1)</code>, averaging over the context length, is both numerically convenient and stable; it is division by a constant and introduces no bias. Once this idiom was carried over to the RL stage, the divisor quietly changed from “the fixed context length” to “the actual length of each response” — from a constant to a variable, and the effect changed with it. Dr.GRPO&rsquo;s implementation fix changes just one line: replace the divisor in <code>masked_mean</code> with a global constant (such as the generation length cap MAX_TOKENS).</p>
<p>The formula&rsquo;s authors and the framework&rsquo;s authors may be running the same set of tests, yet they end up with two different gradients. This kind of “formula–implementation mismatch” is more insidious than paper-level biases: you could read the paper ten times and never catch it; only reading the code reveals it.</p>
<h2 id="drgrpos-fix-and-its-justification">Dr.GRPO&rsquo;s Fix and Its Justification</h2>
<p>The fix is simple: remove both normalizations, <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>1</mn><mi mathvariant="normal">/</mi><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">1/|y_i|</annotation></semantics></math></span> and std, so that the advantage only subtracts the group mean:</p>
<span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><msub><mover accent="true"><mi>A</mi><mo>~</mo></mover><mi>i</mi></msub><mo>=</mo><msub><mi>R</mi><mi>i</mi></msub><mo>−</mo><mrow><mi mathvariant="normal">m</mi><mi mathvariant="normal">e</mi><mi mathvariant="normal">a</mi><mi mathvariant="normal">n</mi></mrow><mo stretchy="false">(</mo><mo stretchy="false">{</mo><msub><mi>R</mi><mi>j</mi></msub><msubsup><mo stretchy="false">}</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mi>G</mi></msubsup><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\tilde{A}_i = R_i - \mathrm{mean}(\{R_j\}_{j=1}^G)</annotation></semantics></math></span><p>Appendix A of the paper gives three theoretical justifications: with both terms removed, the objective <strong>reverts to the PPO objective (Equation 2)</strong>, and the advantage is just the baseline-free Monte Carlo return plus an unbiased baseline; <strong>subtracting the mean introduces no bias</strong> — the baseline&rsquo;s expected contribution to the gradient is identically zero, a standard result in policy gradients; and the corrected advantage differs from RLOO (Leave-One-Out) only by a constant factor <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>G</mi><mi mathvariant="normal">/</mi><mo stretchy="false">(</mo><mi>G</mi><mo>−</mo><mn>1</mn><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">G/(G-1)</annotation></semantics></math></span>, which can be folded into the learning rate and does not affect training dynamics.</p>
<p>The controlled experiments are run in the Oat framework on Qwen2.5-1.5B. The results fall into three layers:</p>
<ul>
<li><strong>Training dynamics</strong>: early on, both GRPO and Dr.GRPO show the R1-Zero-style joint rise of reward and length, but after reward plateaus GRPO&rsquo;s length keeps climbing, while Dr.GRPO&rsquo;s length stabilizes (Fig.5 Plot 1&amp;2);</li>
<li><strong>Behavior</strong>: on benchmarks, the length of Dr.GRPO&rsquo;s incorrect responses drops significantly — the change also mitigates overthinking (Fig.5 Plot 4);</li>
<li><strong>Reproducibility</strong>: after math pretraining, Llama-3.2-3B likewise shows GRPO&rsquo;s length rising together with reward, and Dr.GRPO likewise curbs the continued growth in length (Fig.7). The same recipe also lets the 7B Qwen2.5-Math-7B, using MATH levels 3–5 and training for 27 hours on 8 A100s, reach 43.3% on AIME 2024 — the 7B SOTA at the time, and the whole recipe is extremely simple.</li>
</ul>
<h2 id="a-textbook-erratum-and-the-division-of-labor-with-the-previous-posts">A Textbook Erratum, and the Division of Labor with the Previous Posts</h2>
<p>Section 16.4 of the textbook <a href="https://github.com/walkinglabs/hands-on-modern-rl">hands-on-modern-rl</a>
 also covers this pair of biases, but its wording deserves a correction: it labels “the advantage gets amplified as std approaches zero” as “length bias”, and attributes length inflation to “dividing by std encourages the model to increase within-group variance”. Checked against the original paper, this account is off on two counts: std causes <strong>difficulty bias</strong> (unequal weights across questions), not length bias; and the direct source of length bias is <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>1</mn><mi mathvariant="normal">/</mi><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">1/|y_i|</annotation></semantics></math></span>, which has nothing to do with std and acts on the weight of each token within a response. The two normalizations and the two biases each exist independently — the value of the erratum is that choosing a fix depends on this distinction: removing std does not solve the length problem.</p>
<p>The recent posts in this series happen to tackle this at three different levels:</p>
<ul>
<li><strong>DAPO&rsquo;s token-mean</strong> (§3.3): it changes “average within each sample first, then average over the G samples” to “average all tokens in the batch uniformly by the total token count”, i.e. <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mfrac><mn>1</mn><mrow><msub><mo>∑</mo><mi>i</mi></msub><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow></mfrac><msub><mo>∑</mo><mi>i</mi></msub><msub><mo>∑</mo><mi>t</mi></msub></mrow><annotation encoding="application/x-tex">\frac{1}{\sum_i|y_i|}\sum_i\sum_t</annotation></semantics></math></span>. All tokens are thus weighted by the total token count, which formally eliminates length bias — but DAPO&rsquo;s advantage keeps the z-score (including std), so it <strong>only solves half the problem</strong>;</li>
<li><strong>GSPO</strong>: it uses a single ratio at the sequence level and no longer sums over tokens term by term in the objective, so there is no “divide each token by |y|” averaging step at all — it sidesteps this bias structurally;</li>
<li><strong>Dr.GRPO</strong> (this post): it leaves the structure of the objective unchanged and only removes the two variable divisors from the normalization terms.</li>
</ul>
<h2 id="my-three-takeaways">My Three Takeaways</h2>
<p>First, <strong>an RL training curve must be screened for optimizer-induced false signals before anything else</strong>. When “length keeps rising” is cited by academia as headline evidence that “RL is working” and, at the same time, by engineering teams as an anomaly metric that “calls for intervention”, tracing it back to the root matters all the more. The paper also observes that for Qwen2.5-Math models used with a template, the model&rsquo;s pre-RL ability is first damaged by the template and then rebuilt by RL (§3.3) — another class of factor that needs ruling out: part of the “RL gain” is really just ability that was there all along. Before analyzing an RL curve, rule out, one by one, the biases the optimizer introduces and the artifacts the evaluation setup creates.</p>
<p>Second, <strong>Dr.GRPO and dynamic sampling solve different problems; neither replaces the other</strong>. Removing std normalization corrects weight <strong>distortion</strong> (samples that should not be amplified receive too much weight); DAPO&rsquo;s dynamic sampling (questions with std=0 are simply dropped and resampled) deals with <strong>the absence of gradient signal</strong> (when the responses are all correct/all wrong/all zero, the advantage is still 0 after subtracting the mean, and nothing is learned). Note that after removing std, a zero-signal question is still zero-signal — Dr.GRPO brings no gradient to such questions. One ensures correct relative weights within the samples, the other ensures the samples as a whole carry a learnable signal; a real training system needs both configured at once.</p>
<p>Third, <strong>when checking a normalization, first check “whether the divisor is a variable or a constant”</strong>. The three biases here share one pattern: a normalization that looks like routine practice turns out, on closer inspection, to use a denominator that is not a constant — for <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>1</mn><mi mathvariant="normal">/</mi><mi mathvariant="normal">∣</mi><msub><mi>y</mi><mi>i</mi></msub><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">1/|y_i|</annotation></semantics></math></span> the divisor is the length of each response, for std it is each question&rsquo;s within-group spread, and for <code>masked_mean</code> it is the mask sum of the current batch. Going through every mean/divide in the training code and asking “can this divisor change from sample to sample?” is the cheapest taste audit there is.</p>
<h2 id="open-questions">Open Questions</h2>
<ul>
<li>How the reweighting of “extreme-difficulty questions” after Dr.GRPO removes std plays out in very long training runs (tens of thousands of steps): the time window of the paper&rsquo;s 1.5B/7B experiments is relatively short;</li>
<li>The effective-learning-rate conversion when token-mean (DAPO) and removing the variable divisors (Dr.GRPO) are stacked in the same training stack: there is no public, unified convention;</li>
<li>A systematic check of “how much of the <code>loss.mean(-1)</code> habit from pretraining-era packing still lingers in RL frameworks”: so far there is only the paper&rsquo;s spot check of trl/verl.</li>
</ul>
<h2 id="references">References</h2>
<ul>
<li>Sea AI Lab / NUS: <a href="https://arxiv.org/abs/2503.20783">Understanding R1-Zero-Like Training: A Critical Perspective</a>
 (the source of this post&rsquo;s formulas, experiments, and bias definitions; v2)</li>
<li>Textbook <a href="https://github.com/walkinglabs/hands-on-modern-rl">hands-on-modern-rl</a>
: Section 16.4 (using the corrected wording) and the appendix DAPO cheatsheet (<a href="https://github.com/walkinglabs/hands-on-modern-rl/blob/main/docs/en/appendix_code_cheatsheet/dapo.md"><code>docs/en/appendix_code_cheatsheet/dapo.md</code></a>
, a side-by-side of DAPO&rsquo;s z-score advantage and the token-mean loss granularity)</li>
<li>RLOO reference: <a href="https://arxiv.org/abs/2402.14740">Back to Basics: Revisiting REINFORCE Style Optimization</a>
</li>
<li>Previous posts: <a href="https://njx-njx.github.io/en/posts/dapo-clip-higher-entropy-collapse/">Entropy Collapse: What DAPO&rsquo;s Clip-Higher Fixes</a>
, <a href="https://njx-njx.github.io/en/posts/gspo-sequence-level-importance-sampling/">The Ratio&rsquo;s Wrong Unit: GSPO Goes Sequence-Level</a>
, <a href="https://njx-njx.github.io/en/posts/tis-train-inference-mismatch/">One Model, Two Probabilities: Cost of Mismatch</a>
</li>
</ul>
]]></content:encoded>
    </item>
  </channel>
</rss>
