Product Evals: Travel Planner, Long Context, and The Weight Of Taste
A product note on evaluating an AI travel planner: itinerary quality, OOD scenes, long-context consistency, recommendation taste, and user loops.
Eval
Agentic RL: Reward, Behavior, and The Long Shadow Of Feedback
A field note on agentic RL: reward design, behavior shaping, credit assignment, online and offline evaluation, and the feedback loops behind agents.
Evals As Instruments: Measuring What The Demo Hides
A note on evaluation as an instrument: failure cases, metrics, benchmark design, product loops, and the discipline of measuring agents.