Define “Useful” Before the Research Begins

A research finding can be accurate and still fail to help with the decision. One lesson I keep returning to is to define what “useful” means before collecting evidence—not after seeing which result is easiest to celebrate.

For example, a team testing an AI tool might ask whether its answers are correct. That matters, but the decision may also depend on whether people can verify those answers and finish the task they came to do. Those are related questions, not interchangeable measures.

Writing down the decision, the evidence that could change it, and the limits of the test beforehand makes the conclusion easier to interpret. It also makes an inconclusive result informative: perhaps the tool needs improvement, or perhaps the evaluation didn’t capture the work that matters.

A small discipline with strategic value: decide what the evidence is meant to settle before it arrives.