<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>engineering on Njx&#39;Log</title>
    <link>https://njx-njx.github.io/en/tags/engineering/</link>
    <description>Recent content in engineering on Njx&#39;Log</description>
    <image>
      <title>Njx&#39;Log</title>
      <url>https://njx-njx.github.io/blog-cover.jpg</url>
      <link>https://njx-njx.github.io/blog-cover.jpg</link>
    </image>
    <generator>Hugo</generator>
    <language>en</language>
    <lastBuildDate>Tue, 27 Oct 2026 15:14:17 +0800</lastBuildDate>
    <atom:link href="https://njx-njx.github.io/en/tags/engineering/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Anatomy of a Minimal SFT Trainer: Every Decision</title>
      <link>https://njx-njx.github.io/en/posts/forge-sft-anatomy/</link>
      <pubDate>Fri, 28 Aug 2026 20:12:53 +0800</pubDate>
      <guid>https://njx-njx.github.io/en/posts/forge-sft-anatomy/</guid>
      <description>Why hand-write an SFT loop when TRL trains in one line? Seven decisions in a minimal trainer, checked against textbooks and production frameworks.</description>
      <content:encoded><![CDATA[<p>TRL&rsquo;s SFTTrainer can train in a single line, so why does anyone still hand-write an SFT loop? I put every Stage 1 decision in the forge repo on the table one by one (chat template, prompt masking, collate and loss, bf16, LR schedule, data, validation) and check each against two textbooks (rlhf-book, hands-on-modern-rl), four production frameworks (alignment-handbook, trl, open-instruct, OpenRLHF), and six information sources: which decisions have consensus, which are genuine recipe disputes, and which the two textbooks each gloss over.</p>
<blockquote>
<p>This post comes with a companion <a href="https://njx-njx.github.io/reports/forge-sft-anatomy/">interactive report</a>
 (in Chinese), which includes a decision comparison matrix and a drill-down index of sources.</p>
</blockquote>
<h2 id="why-hand-write-a-trainer-of-only-511-lines">Why Hand-Write a Trainer of Only 511 Lines</h2>
<p>I&rsquo;m building forge, a post-training repo that grows stage by stage along my learning path (Stage 1: SFT; local, not yet pushed as open source). It has only one principle: use mature engineering components off the shelf, but write every line of the core training-data path myself and be able to explain why.</p>
<p>There are three routes to writing SFT. The hands-on-modern-rl course&rsquo;s <a href="https://github.com/walkinglabs/hands-on-modern-rl"><code>sft_pipeline.py</code></a>
 is a one-line call to TRL&rsquo;s <code>SFTTrainer</code>, with the training loop, loss, and padding all inside the library; the <a href="https://github.com/natolambert/rlhf-book"><code>instruction_tuning</code></a>
 code from Nathan Lambert&rsquo;s &ldquo;RLHF Book&rdquo; is a hand-written loop; production frameworks (alignment-handbook, open-instruct, OpenRLHF) stack a whole suite of packing, flash-attention, and decontamination on top of the Trainer.</p>
<p>forge went with hand-writing. The motivation isn&rsquo;t performance—a 0.5B model trains fast no matter how you train it—but that <strong>every decision must be explicitly visible</strong>, because every later stage (RM, DPO, GRPO, agentic RL) will reuse this loop, and the RL stages will also have to stuff sampling and reward into it; the black box has to be opened sooner or later. This post lines up each of forge&rsquo;s decisions against two textbooks, four production frameworks, and six information sources (Notion notes, Zhihu, a Feishu industry-news group, arXiv, local video courses). Conclusion first: most decisions have strong consensus, a few are genuine recipe disputes, and then there are a few pitfalls that the two textbooks each paper over in different ways.</p>
<h2 id="decision-1-chat-templatethree-pitfalls-in-string-concatenation">Decision 1: Chat Template—Three Pitfalls in String Concatenation</h2>
<p>The raw data is structured messages (role + content), but the model only understands plain text, so the first step is rendering. forge hand-writes ChatML: each message is wrapped as <code>&lt;|im_start|&gt;{role}\n{content}&lt;|im_end|&gt;\n</code>. I verified that this is <strong>character-for-character identical</strong> to the output of the <code>chat_template</code> bundled with Qwen&rsquo;s official tokenizer.</p>
<p>It looks like there&rsquo;s no real decision in this, but three sources each turned up a pitfall:</p>
<ol>
<li><strong>Base models have no template</strong>. OLMo-2-1B is a base model, and its tokenizer has no chat template in it; rlhf-book&rsquo;s approach is to copy the entire <code>chat_template</code> attribute from the tokenizer of the sibling model <code>allenai/OLMo-2-0425-1B-SFT</code> (<code>utils.py:82-88</code>). In the real world, a template isn&rsquo;t necessarily ready-made.</li>
<li><strong><code>apply_chat_template</code> has hidden behavior</strong>. When the messages have no system turn, Qwen&rsquo;s template injects a default system prompt (&ldquo;You are Qwen, created by Alibaba Cloud&hellip;&rdquo;). The hands-on course&rsquo;s data has no system turn, so its training text actually carries this default system line—using the tokenizer&rsquo;s bundled template can bring in text you never reviewed. forge renders only the turns already present in the sample and injects nothing.</li>
<li><strong>Template errors are SFT&rsquo;s number-one silent problem</strong>. HuggingFace&rsquo;s official blog published &ldquo;Chat Templates: An End to the Silent Performance Killer&rdquo; (2024-01; my Notion materials library tags it as &ldquo;the most frequent SFT pitfall&rdquo;); and the first item on the Chinese community&rsquo;s Trainer troubleshooting checklist is &ldquo;loss not going down → wrong chat template / wrong labels mask&rdquo;.</li>
</ol>
<p>My trade-off: hand-write the template first to understand its structure (a template is just string concatenation; the special tokens are registered in the vocabulary, so <code>&lt;|im_start|&gt;</code> encodes to exactly one whole token and doesn&rsquo;t fragment), so that later, when I hook into the official template, I know what I&rsquo;m hooking into.</p>
<h2 id="decision-2-prompt-maskingthe-biggest-point-of-contention-in-this-post">Decision 2: Prompt Masking—The Biggest Point of Contention in This Post</h2>
<p>For the rendered sequence, every position needs a decision on whether it counts toward the loss. The rule is: <strong>the logits at each position predict the next token; if the prediction target is an assistant token, it is scored, otherwise it is set to -100</strong>. forge&rsquo;s choice is to count every assistant turn (including the wrapping markers) and mask all user/system turns.</p>
<p>How this decision is distributed across the sources is interesting:</p>
<ul>
<li><strong>rlhf-book trains only on the final assistant turn</strong> (assistant replies in the middle of a multi-turn conversation are masked too), and even masks the <code>&lt;|assistant|&gt;</code> generation header itself (<code>utils.py:138,142-148</code>). Chapter 4 of its textbook explicitly lays out both conventions side by side: train only on the final turn, or mask only the user turns. forge chose the latter.</li>
<li><strong>The hands-on course&rsquo;s code doesn&rsquo;t mask at all</strong>: it feeds SFTTrainer a &ldquo;text&rdquo; column and computes the loss over the entire sequence. Yet the course&rsquo;s own docs (<code>imitation-learning-pipeline.md:236-238</code>) say that counting user/system tokens in the loss is &ldquo;the most common pitfall&hellip; this isn&rsquo;t pedantry; it&rsquo;s protecting the model you train from reciting the prompts&rdquo;. <strong>Same repo, and the code and the docs contradict each other</strong>—the most direct example of &ldquo;teaching implementation ≠ best practice&rdquo;.</li>
<li><strong>trl&rsquo;s current approach</strong> is <code>assistant_only_loss</code> + <code>{% generation %}</code> markers in the template, with an explicit warning: if an assistant turn&rsquo;s end token isn&rsquo;t inside the loss mask, &ldquo;the model may never learn to stop&rdquo;. This corroborates the need for forge to train on <code>&lt;|im_end|&gt;</code>—the model must learn to stop.</li>
<li><strong>Ownership of the boundary tokens</strong>: production templates (trl/OpenRLHF) usually wrap only the assistant content in the generation markers, i.e., the <code>&lt;|im_start|&gt;assistant\n</code> header doesn&rsquo;t count toward the loss; forge counts the header too. A difference of a token or two—a detail open to debate.</li>
<li><strong>Masking itself has been challenged by research</strong>: &ldquo;Instruction Tuning with Loss over Instructions&rdquo; (arXiv <a href="https://arxiv.org/abs/2405.14394">2405.14394</a>
) found that also computing the loss on the instruction part (IM) outperforms computing it only on the output (OM) in many settings across 21 benchmarks. In other words, completion-only is the current framework default, not something proven correct—when the goal is for the model to internalize the instruction distribution, masking throws away signal.</li>
</ul>
<p>Written out as a formula, forge&rsquo;s loss is:</p>
<span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mi>L</mi><mo>=</mo><mo>−</mo><mfrac><mn>1</mn><mrow><mi mathvariant="normal">∣</mi><mi>A</mi><mi mathvariant="normal">∣</mi></mrow></mfrac><munder><mo>∑</mo><mrow><mi>t</mi><mo>:</mo><mtext> </mtext><msub><mi>x</mi><mrow><mi>t</mi><mo>+</mo><mn>1</mn></mrow></msub><mo>∈</mo><mi>A</mi></mrow></munder><mi>log</mi><mo>⁡</mo><msub><mi>p</mi><mi>θ</mi></msub><mo stretchy="false">(</mo><msub><mi>x</mi><mrow><mi>t</mi><mo>+</mo><mn>1</mn></mrow></msub><mo>∣</mo><msub><mi>x</mi><mrow><mo>≤</mo><mi>t</mi></mrow></msub><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">L = -\frac{1}{|A|}\sum_{t:\,x_{t+1}\in A}\log p_\theta(x_{t+1}\mid x_{\le t})</annotation></semantics></math></span><p>where <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi>A</mi></mrow><annotation encoding="application/x-tex">A</annotation></semantics></math></span> is the set of tokens in assistant turns and <span class="katex"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mi mathvariant="normal">∣</mi><mi>A</mi><mi mathvariant="normal">∣</mi></mrow><annotation encoding="application/x-tex">|A|</annotation></semantics></math></span> is the number of valid tokens.</p>
<h2 id="decision-3-collate-and-lossline-for-line-isomorphic-plus-a-tail-neither-book-handles-well">Decision 3: Collate and Loss—Line-for-Line Isomorphic, Plus a Tail Neither Book Handles Well</h2>
<p>The core trio on the training side: right-pad collate (pad tokens fill the inputs, -100 fills the labels, attention_mask is marked 0), shift (<code>logits[:, :-1]</code> against <code>labels[:, 1:]</code>), and <code>F.cross_entropy(ignore_index=-100)</code>. This implementation is line-for-line isomorphic to rlhf-book&rsquo;s—two independently written pieces of code that look the same are the strongest evidence that &ldquo;this is the standard practice&rdquo;.</p>
<p>But the side-by-side comparison exposed two problems that not even the textbooks handle cleanly:</p>
<ol>
<li><strong>The grad accumulation tail</strong>. The number of micro-batches in an epoch isn&rsquo;t necessarily divisible by the number of accumulation steps. rlhf-book simply discards the leftover group at the tail—the backward passes are wasted, and those few samples contribute nothing to the update; forge steps on the leftover group as well, but still divides the loss by the nominal accum count—so the leftover group&rsquo;s gradient is applied at a scale that&rsquo;s too small. Each of the two books fudges half of it. What&rsquo;s more common in real production is to normalize by the leftover group&rsquo;s actual size, or simply set <code>drop_last=True</code> and align the scheduler&rsquo;s step count.</li>
<li><strong>The micro-batch averaging convention</strong>. Both forge and rlhf-book &ldquo;average within each micro-batch first, then divide by accum&rdquo;. When samples differ greatly in their number of trainable tokens, this differs subtly from &ldquo;a weighted average over all tokens&rdquo;. The vast majority of frameworks accept this compromise, but if you want to pick at training details (mixed data with large length differences), this is worth coming back to. The production-side counterpart is trl&rsquo;s <code>chunked_nll</code>: drop the -100 positions first, then compute the projection—mathematically equivalent, and it saves peak GPU memory.</li>
</ol>
<p>OpenRLHF has another detail worth copying: <strong>sequences are forced to end with EOS</strong>—after truncation, <code>input_ids[-1] = eos_token_id</code> (<code>sft_dataset.py</code>). forge makes no such guarantee: the last token of a truncated sample is arbitrary, and when truncation happens to cut off the <code>&lt;|im_end|&gt;</code> at the end of an assistant turn, that amounts to teaching the model to &ldquo;stop halfway through&rdquo;.</p>
<h2 id="decision-4-precisionpure-bf16-and-a-redundant-layer-of-autocast">Decision 4: Precision—Pure bf16 and a Redundant Layer of Autocast</h2>
<p>Both textbooks load the weights in pure bf16: no autocast, no fp32 master weights, no GradScaler. Teaching code goes for convenience, and a 0.5B model really doesn&rsquo;t call for more anyway.</p>
<p>forge&rsquo;s approach exposes something I need to correct: the weights are already loaded in bf16 (<code>model.py</code>), yet the forward pass is additionally wrapped in a <code>torch.autocast(bf16)</code> (<code>train.py:92</code>)—when the weights are already bf16, this autocast layer is essentially redundant. Real mixed precision (fp32 master + loss scaling) lives at the Megatron/DeepSpeed level; Stage 1 doesn&rsquo;t need it, but it deserves a note in the blog. Tulu 3&rsquo;s <code>--mixed_precision bf16</code> and the community&rsquo;s &ldquo;BF16 is 2x faster than FP32&rdquo; (hands-on course docs) give this default production-side backing.</p>
<h2 id="decision-5-lr-schedulecosine-vs-linear-is-a-recipe-dispute">Decision 5: LR Schedule—Cosine vs. Linear Is a Recipe Dispute</h2>
<p>forge uses <code>get_cosine_schedule_with_warmup</code> with 3% warmup. Production gives no single answer: Tulu 3&rsquo;s 8B SFT recipe is <strong>lr 5e-6 + linear decay + warmup 0.03</strong> (open-instruct <code>finetune_8b.sh</code>; a reproducible config is harder evidence than a paraphrase of the paper); Zephyr uses 2e-5 + cosine + warmup 0.1; OpenRLHF defaults to cosine_with_min_lr. rlhf-book simply hand-writes a linear decay—the teaching book itself doesn&rsquo;t use cosine either.</p>
<p>Two worthwhile observations:</p>
<ul>
<li><strong>The default scheduler is a trap</strong>. If you don&rsquo;t specify one explicitly, HF/TRL default to linear + warmup 0 (that&rsquo;s exactly how the hands-on course&rsquo;s code was run). You think you didn&rsquo;t make a decision, but you already did.</li>
<li><strong>lr and packing are strongly coupled</strong>. The OLMo 3 practice cited in the textbook: sequence packing significantly increases the effective token batch, so the lr has to go up with it. Talking about lr in isolation from the throughput configuration is meaningless.</li>
</ul>
<p>forge&rsquo;s numbers match mainstream recipes item by item: lr 1e-5 (within the 5e-6~2e-5 production range, and in line with &ldquo;the smaller the model, the larger the lr&rdquo;), warmup 0.03, weight decay 0.0, grad clip 1.0, epochs 2. An effective batch of 32 (8×4) is teaching-scale for 0.5B—rlhf-book coincidentally also uses 32 (4×8), while production uses 128-256. epochs=2 has research-side support: SFT scaling research shows that the gains from repeated data come roughly within the first 4 epochs (arXiv <a href="https://arxiv.org/abs/2402.17193">2402.17193</a>
), and &ldquo;SFT Memorizes, RL Generalizes&rdquo; (arXiv <a href="https://arxiv.org/abs/2501.17161">2501.17161</a>
) further points out that training for more epochs mainly reinforces memorization rather than generalization—2 epochs is in the safe zone, and it also foreshadows the ceiling of a pure SFT slice.</p>
<h2 id="decision-6-dataforges-biggest-simplification-reveals-the-full-production-picture">Decision 6: Data—forge&rsquo;s Biggest Simplification Reveals the Full Production Picture</h2>
<p>forge uses 10,000 examples from the <code>everyday-conversations</code> subset of <code>HuggingFaceTB/smoltalk</code>: a single source, zero quality filtering. This is the biggest simplification in the entire repo, and the sources pin down its place precisely:</p>
<p>The SmolTalk dataset card (verified) states: the full set has 1.1M examples in 14 subsets, and the core new data is Smol-Magpie-Ultra 400K (Magpie pipeline + Llama-3.1-405B generation); <strong>everyday-conversations is multi-turn everyday dialogue, merely a &ldquo;basic conversational behavior&rdquo; slice</strong>, with math, code, and long context covered by other subsets. In other words, training on it alone yields the conversational layer of the full recipe, not full capability.</p>
<p>Both ends of the data-scale spectrum have names: LIMA (arXiv <a href="https://arxiv.org/abs/2305.11206">2305.11206</a>
) used 1,000 hand-curated examples to bring a 65B model close to the GPT-4 of its day and proposed the &ldquo;Superficial Alignment Hypothesis&rdquo;—pretraining has already stored up the capabilities, and SFT only does shallow alignment; Tulu 3 (arXiv <a href="https://arxiv.org/abs/2411.15124">2411.15124</a>
) used 940K examples for rigorous multi-skill <strong>mixture ablations</strong>, backed by a decontamination toolchain. In between sit AlpaGasus (90% of the 52K was low quality; the 9K filtered out of it beat the full set) and Deita (scoring on complexity, quality, and diversity; 6-10K beat larger mixed sets). One line in my Notion notes sets the tone for this spectrum: LIMA coexists with industrial million-scale datasets, and every hard capability needs data behind it; data work comes down to just four decisions—what to get, what to keep, how much to mix in, and what to do when it isn&rsquo;t enough.</p>
<p>As for what production data engineering actually looks like, the Chinese community and the Feishu group supplied the two ends: a Zhihu column calls the curation pipeline (dedup → quality gate → remove overlap with eval sets → ensure multi-turn completeness) &ldquo;the link in the whole pipeline that deletes the most and is the least transparent&rdquo;; on mixture ratios, mixing in 10%-20% general data to prevent catastrophic forgetting is the highest-consensus rule of thumb in the community (there are also two claims a factor of ten apart, 1:1 and 1:10~1:100 experience replay—the ratio itself is unsettled). Claims like &ldquo;90% of SFT performance depends on data quality&rdquo; circulate widely in the community, but no experimental source can be given for them, so they can only be treated as a sentiment indicator.</p>
<p>Newer signals come from the industry-news group: Shanghai AI Lab&rsquo;s SKT, AutoCompact&rsquo;s judge-guided SFT, MindForge&rsquo;s trajectory fine-tuning—production SFT data is increasingly &ldquo;strong-model trajectories + verifiers&rdquo;, and the smoltalk that forge loads directly is, in essence, also trajectories that someone else has already distilled. Another paper shared in the group, &ldquo;SFT Conflicts, RL Coexists&rdquo; (arXiv <a href="https://arxiv.org/abs/2608.03573">2608.03573</a>
), supplies a footnote from the opposite direction: multi-stage multi-task SFT degrades by 23.1% on average, while the same multi-stage RL improves by 24.9%—the pitfall a minimal single-task, single-stage SFT sidesteps isn&rsquo;t in the loop itself but in the data composition.</p>
<p>My judgment: 10,000 examples from a single source is a reasonable teaching starting point—first verify that the pipeline is correct; mixtures and quality gates are Stage 2&rsquo;s business. One practical constraint to keep firmly in mind: right now <code>max_len=1024</code>, while long reasoning-chain data routinely runs 2000-8000 tokens, so when switching data the context length has to change along with it; otherwise a large number of samples will be truncated into &ldquo;the question plus half a thinking process&rdquo;, with <code>&lt;|im_end|&gt;</code> cut off in the bargain.</p>
<h2 id="decision-7-validationval-loss-is-the-low-variance-choice-but-only-half-the-story">Decision 7: Validation—Val Loss Is the Low-Variance Choice, but Only Half the Story</h2>
<p>forge&rsquo;s validation trio: hold out 1% of the data as a val split, compute val loss after training, and greedy-decode 8 fixed prompts to eyeball the results.</p>
<p>A methodological rule in my Notion notes directly backs val loss: accuracy-type metrics have high variance on small models, so <strong>prefer per-domain validation loss, repeated across multiple seeds</strong>. Run an MMLU-style eval on a 0.5B model and the noise will drown out the signal; loss is at least stable.</p>
<p>The course docs (unreviewed, but this point matches frontline experience) list five metrics to watch in SFT: train_loss, eval_loss, format pass rate, repetition rate, and manual spot checks, and warn that &ldquo;looking only at loss is a beginner&rsquo;s mistake—the loss will look great, but the model will turn into an assistant with correct formatting and anemic content&rdquo;. rlhf-book&rsquo;s approach supplies the other half: during training it periodically generates samples from 4 fixed prompts, and at step 0 it first prints the base model&rsquo;s outputs—the qualitative contrast of &ldquo;base misses the question → after SFT it answers and then stops&rdquo; is the most intuitive way to check whether a checkpoint is taking effect. Ironically, the entire module never saves the model (verified with grep): it has a tracker and in-training generation, but no checkpoint.</p>
<p>What forge currently lacks: periodic generation during training, quality metrics like format pass rate, and a wandb tracker. Plus a set of throughput optimizations from production practice: packing + padding-free + flash-attention (Netflix&rsquo;s write-up reports 4.7x throughput, and IBM&rsquo;s arXiv <a href="https://arxiv.org/abs/2407.09105">2407.09105</a>
 is the authoritative account of putting it into production), length-bucketed batching, and gradient checkpointing on the training side. None of these go into Stage 1—the point of minimalism is to let you see clearly what the system is doing without them.</p>
<h2 id="the-not-done-list-is-the-stage-2-roadmap">The Not-Done List Is the Stage 2 Roadmap</h2>
<p>Having compared against all six sources, here&rsquo;s what forge deliberately left out and what it should add soon, in order of priority:</p>
<ul>
<li><strong>P0: packing + padding-free + FA2</strong>. All four production frameworks have it, and it&rsquo;s the number-one engineering gap between forge and production; Netflix&rsquo;s 4.7x throughput shows this is no mere detail.</li>
<li><strong>P1: align the mask boundaries with the production convention</strong> (assistant header excluded from the loss, end token always included), and <strong>force truncated samples to end with EOS</strong> (a one-line fix, as in OpenRLHF).</li>
<li><strong>P1: normalize the accum tail by its actual group size</strong>; and while at it, remove that redundant autocast layer.</li>
<li><strong>P2: chunked loss, grad checkpointing, decontamination</strong>—these can wait for the large-model stage; decontamination belongs to the data-preparation layer and doesn&rsquo;t go into the trainer.</li>
<li><strong>Not pursuing: NEFTune</strong> (frameworks support it, mainstream recipes don&rsquo;t use it), <strong>switching cosine to linear</strong> (a recipe dispute with two coexisting camps; forge sides with the majority).</li>
</ul>
<p>Looking back, what hand-writing 511 lines bought me isn&rsquo;t performance but a decision checklist: why each knob sits where it does, how the textbooks choose, how production chooses, and what research says. SFTTrainer runs in one line, but that one line hides a dozen-plus default decisions, three of which (full-sequence loss, linear schedule with zero warmup, silent truncation) happen to be among the top entries on the community&rsquo;s troubleshooting checklists. Only by taking the defaults apart do you know what you&rsquo;re choosing.</p>
<h2 id="references-and-sources">References and Sources</h2>
<p><strong>Textbook code</strong>: rlhf-book <code>code/instruction_tuning/</code> (<a href="https://github.com/natolambert/rlhf-book">repo</a>
; key line numbers are given in the main text); hands-on-modern-rl <code>code/chapter08_rlhf/sft_pipeline.py</code> and <code>docs/chapter15_rlhf/</code> (<a href="https://github.com/walkinglabs/hands-on-modern-rl">repo</a>
; the course states that it has not been comprehensively reviewed).</p>
<p><strong>Production frameworks</strong> (all main-branch source snapshots): <a href="https://github.com/huggingface/alignment-handbook">alignment-handbook</a>
 <code>recipes/zephyr-7b-beta/sft/config_full.yaml</code>; <a href="https://github.com/huggingface/trl">trl</a>
 <code>sft_trainer.py</code>/<code>sft_config.py</code>; <a href="https://github.com/allenai/open-instruct">open-instruct</a>
 <code>finetune.py</code> and <code>scripts/train/tulu3/finetune_8b.sh</code>; <a href="https://github.com/OpenRLHF/OpenRLHF">OpenRLHF</a>
 <code>openrlhf/datasets/sft_dataset.py</code>.</p>
<p><strong>Papers</strong>: LIMA <a href="https://arxiv.org/abs/2305.11206">2305.11206</a>
 · AlpaGasus <a href="https://arxiv.org/abs/2307.08701">2307.08701</a>
 · Deita <a href="https://arxiv.org/abs/2312.15685">2312.15685</a>
 · Tulu 3 <a href="https://arxiv.org/abs/2411.15124">2411.15124</a>
 · SmolLM2 <a href="https://arxiv.org/abs/2502.02737">2502.02737</a>
 · Magpie <a href="https://arxiv.org/abs/2406.08464">2406.08464</a>
 · SFT scaling <a href="https://arxiv.org/abs/2402.17193">2402.17193</a>
 · SFT Memorizes, RL Generalizes <a href="https://arxiv.org/abs/2501.17161">2501.17161</a>
 · Instruction Modelling <a href="https://arxiv.org/abs/2405.14394">2405.14394</a>
 · Packing with FlashAttention <a href="https://arxiv.org/abs/2407.09105">2407.09105</a>
 · SFT Conflicts, RL Coexists <a href="https://arxiv.org/abs/2608.03573">2608.03573</a>
.</p>
<p><strong>Community and industry</strong>: <a href="https://huggingface.co/blog/chat-templates">HuggingFace · Chat Templates: An End to the Silent Performance Killer</a>
 · <a href="https://zhuanlan.zhihu.com/p/2073360093089883686">Zhihu · SFT data curation</a>
 · <a href="https://zhuanlan.zhihu.com/p/2023522185143173700">Zhihu · Training data preparation for SFT and RL</a>
 · <a href="https://netflixtechblog.com/">Netflix engineering blog · Scaling LLM Post-Training</a>
 (indexed via the Notion materials library, 2026-02) · Feishu industry-news group (posts on SKT, AutoCompact, MindForge, 2026-05 to 2026-08).</p>
<p><strong>forge repo</strong>: <code>src/forge/data.py</code>, <code>src/forge/train.py</code>, <code>src/forge/model.py</code>, <code>configs/sft.yaml</code>, <code>tests/test_data.py</code> (local, not yet pushed as open source; line numbers in the text refer to the 2026-08-28 working tree).</p>
]]></content:encoded>
    </item>
  </channel>
</rss>
