Change the Prompt, Keep the Comparison

A useful lesson from thinking through AI workflow changes: you can’t tell what improved if you lose the comparison point.

Imagine revising a prompt that supports a team’s review process. Before switching over, keep a small set of representative inputs and the current outputs—not as a claim that they’re perfect, but as a record of behavior people already rely on. Run the revised version against the same cases. Notice what improved, what regressed and what remains uncertain.

Then change one part of the workflow, keep the old version available as a fallback, and review the results with the people who use them. If the new version creates confusion, restore the prior one while you investigate. A fresh prompt is easy to write; recovering a useful comparison after it’s gone is less so.