Give an AI knowledge base a retirement rule

A knowledge base is also a maintenance commitment. Here is a hypothetical exercise for an AI assistant that answers questions about an internal process—not a report of a deployed system.

Imagine a team changes its request form. The old instructions still read clearly, and the assistant can repeat them accurately, but they now send people to a retired form. Fluency is not the issue; the source has outlived its useful scope.

Try a small source-care record:

• Scope: which question does this document answer, and for whom?
• Steward: who has agreed to maintain it, with what time available?
• Review trigger: a form, policy or owner changes—not only a calendar date.
• Retirement rule: when should the assistant stop using it, and where should unresolved questions go?
• Reader notice: explain what changed and what remains uncertain in language the affected person can use.

For a tabletop test, retire one fictional document while leaving a similar valid document available. Check whether the assistant excludes the retired source, preserves the valid one and explains a missing answer honestly. Keep a record of the source version and the reviewer’s decision; do not turn “recently reviewed” into a guarantee that every claim is correct.

The trade-off is upkeep. If nobody can own a high-impact source, narrowing the assistant’s scope may be more sustainable than adding more documents.

Which signal should trigger retirement immediately, and which should trigger human review?