Build evaluation datasets

Datasets make a quality decision repeatable. Select traces that represent common requests, known failures, and important edge cases, then preserve the input, expected behavior, and evaluation context.

Keep the dataset stable while comparing prompt or model candidates. A useful dataset is small enough to run frequently and broad enough to expose regressions that a single happy-path example cannot show.

Run the dataset through the evaluation engine, then compare quality, cost, latency, and failure rates before changing a production label.