Jiaxing Ni
19, working on the engineering side of LLM training — harness, evals, post-training, and every pipeline where the three meet.
Good systems feel quiet on the surface. Underneath: rollout pipelines, reward traces, eval slices, and a few scripts watching the night shift. Most of my work is making those things tell the truth.
What I Work On
One through-line: turn real-world usage traces into signals that training can learn from.
- Dig model failure patterns out of production sessions and user feedback, then forge them into benches and multi-turn data
- Build executable verifiers for agent tasks — layered scoring gates, mutation-test self-checks, judging noise that can be peeled out
- An eval owes more than a score; it should answer why it generalizes
- RL environments, rollout systems, SFT data synthesis — every gram of post-training signal should have a provenance
Traces
Every stop so far circles the same problem: agent harness and swarm evals, distilling a model’s recurring failure modes out of a hundred-thousand-odd multi-turn sessions; a pretraining data foundation, plus an attempt to give a base model the “taste” to judge training recipes; the mechanics of evaluation itself — scores anchored to evidence, user satisfaction made interpretable through the geometry of a conversation; and further back, a system that gathered personal context scattered across macOS into something trustworthy, permissioned, and actionable.
Which banners these happened under, I’ll leave off the page — the work matters more than the names. Details are for offline conversation.
Coordinates
- Identity: researcher, builder, engineer.
- Base: Beijing / Shanghai.
- Taste: prototype quickly, measure honestly, keep only what survives contact with real edges.
About This Blog
A field notebook on the mechanisms behind LLM training: what a knob actually does, what breaks when you turn it, and what the metric said.
Topics orbit harness and rollout systems, model architecture, mid-training and post-training, evals and benchmarks, reasoning RL and agentic RL, RSI, and data synthesis. Longer posts often come with an interactive report edition, where formulas become sliders and curves become probes — see Reports.
Built with Hugo and PaperMod, hosted on GitHub Pages, and open source on GitHub.
FAQ
What will you not write here? I try not to write generic AI commentary. If a post has no code, no experiment trace, no eval detail, or no honest uncertainty, it probably belongs somewhere else.
Can I cite, translate, or share a post? Yes. Keep a link to the original post, and feel free to tell me where it traveled. I like seeing notes find another desk.
Are you open to collaboration or conversation? Yes, especially around RL infrastructure (rollout and serving), post-training recipes, eval design, agentic systems, and strange little tools that make research easier to use.