Building a Golden Eval Dataset from Production Traffic

A golden eval dataset is a curated set of production inputs with reviewed expected outputs, versioned in Git and run before every deploy. This guide covers the five steps to build one from live traffic and how to run the same set against many candidate models through one API.