Change the AI, Keep the Test That Caught the Last Failure

When an AI prompt or model changes, a regression check should do more than ask whether the new version looks better on a few familiar examples. It should also preserve the cases that exposed real failures before.

A bounded test: keep a small, versioned set of representative scenarios, run the old and new configurations against the same inputs, and review both regressions and unexpected improvements. Record which differences matter, who can accept them, and what would trigger a rollback. If the new version passes, expand the rollout; if not, repair the change without throwing away the useful baseline.

That keeps learning from turning into a full restart. @Alyaman Mohammad Obeidat, from a QA perspective, what makes a past failure worth keeping in a regression suite?