What is the role of golden datasets in AI evaluation?

A golden dataset is a curated, trusted set of inputs paired with known-good expected outputs (or acceptance criteria) that represents your real use cases and important edge cases. It's the fixed yardstick you evaluate every prompt and model change against, so results are comparable over time and a regression is immediately visible. Good practice: keep it representative (mirror real query distribution), include hard and adversarial cases, version it, and grow it whenever a production bug reveals a gap the set missed. Keep it out of any training or few-shot data to avoid contaminating the measurement. A small, high-quality golden set that you actually run in CI beats a huge, stale one nobody trusts.