Debug RAG at the Retrieval–Generation Handoff

A proposed debugging exercise for a fictional retrieval-augmented generation (RAG) assistant: separate “Did it find the right evidence?” from “Did it use that evidence well?”

Suppose someone asks, “What is the cancellation window?” Before revising the prompt, inspect the retrieved passages for that query:

• If the applicable policy passage is missing, investigate retrieval.
• If it is present but the answer misstates or omits it, investigate how the answer uses its context.
• If sources conflict or none applies, mark the case ambiguous rather than forcing it into either category.

Keep the query, expected passage and version, retrieved passages, and final answer as separate review fields. That makes it easier to see which part of the pipeline needs attention before changing settings or instructions.

This is a hypothetical diagnostic, not a report of measured results. Which handoff would you inspect first in a RAG workflow: retrieval, context use, or source-version handling?