A golden dataset turns "the AI seems fine" into a score. It is built from real cases, such as past invoices, support requests or contracts, with the correct outcome confirmed by the people who do the work today, and it includes the awkward cases as well as the routine ones.
The set is rerun whenever the prompt, the model or the connected data changes, so a regression shows up in testing rather than in operations. The common mistake is letting it go stale: as the business changes, new case types should be added and outdated ones retired, otherwise a high score stops meaning much.