An AI answer should earn the right to avoid saying “I don't know”

A factuality evaluation should show which questions were answered, which were wrong and when missing evidence was acknowledged.

An AI answer should earn the right to avoid saying “I don't know”

A research assistant returns a supplier's opening date with a confident sentence and a percentage beside it. The date is plausible, but the supplied documents do not contain it. The reader now has two unsupported claims: the date and the apparent confidence. A useful evaluation must ask when the assistant should answer, when it should ask for better evidence and when it should stop. Counting only completed answers makes that decision easy to miss.

Prepared and checked on 17 October 2026, this analysis compares three dated primary research accounts: uncertainty over answer meanings, incentives created by scoring, and different kinds of factuality tests. It proposes an evaluation for a fictional document-research service. It is not a claim to have tested current models, identified a best product or eliminated incorrect answers.

Three ideas that answer different questions

Farquhar and colleagues' June 2024 Nature paper develops semantic entropy to detect a subset of hallucinations they call confabulations: answers that are both incorrect and sensitive to arbitrary generation variation. The method compares meanings across generated answers rather than treating every wording change as a different answer. The paper distinguishes this from consistently repeated false beliefs and other systematic errors. [Author paper in Nature, available through PMC](https://pmc.ncbi.nlm.nih.gov/articles/PMC11186750/)

OpenAI's 5 September 2025 research account argues that evaluation incentives can favour guessing over acknowledging uncertainty. Its analysis separates accuracy from other outcomes such as errors and abstentions. This is an attributed research argument about training and evaluation, not proof that a particular model's refusal is calibrated or that a new scoring rule fixes all factual failures. [OpenAI research account](https://openai.com/index/why-language-models-hallucinate/)

Google DeepMind's 9 December 2025 FACTS Benchmark Suite distinguishes parametric knowledge, search, multimodal factuality and grounding in provided context. Its grounding update tests responses against supplied material. These are different evaluation settings, and a score in one should not be treated as a universal probability that an answer in another setting is true. [FACTS Benchmark Suite](https://deepmind.google/blog/facts-benchmark-suite-systematically-evaluating-the-factuality-of-large-language-models/)

These approaches can inform a test plan, but they are not interchangeable safety guarantees. Meaningful agreement among several answers, faithfulness to a document and correctness about the world describe different relationships. A model may repeat a wrong date consistently. A response may accurately reproduce a document that is itself outdated. A search result may be current but irrelevant to the exact organisation being discussed.

Define the answerable task

For the fictional research service, begin with a narrow question: can an assistant extract specified facts from a reviewed set of documents and identify when the evidence is insufficient? Record the allowed source set and its dates. Do not let a model quietly switch from document extraction to unsupported recollection when the answer is missing.

Build cases where the correct behaviour differs. Some documents contain an explicit answer. Others contain a related date, an ambiguous organisation name, conflicting versions or no answer at all. Include a case where a document clearly supports a statement but is too old for the user's question. A well-designed test needs missing information as well as easy positive examples.

Have qualified reviewers establish the expected evidence and acceptable responses before comparing systems. An abstention can be appropriate when the answer is absent, but unhelpful when the document states it clearly. A clarification question can be useful when it resolves a real ambiguity. Treating every refusal as safe would reward a service that never performs its task.

Count answers, errors and withheld answers together

Report how many questions the system answered and how many of those answers were correct. Also report correct abstentions, unnecessary abstentions and unsupported answers. This prevents an apparent quality improvement from being explained entirely by answering fewer questions. Separate results by task type: a model that handles explicit dates well may still struggle with conflicting records.

If the system provides a confidence score, test what that score means on held-out examples. Decide whether it concerns document support, factual correctness or some other event. Compare score groups with reviewer outcomes and state the sample size and uncertainty. A percentage generated in prose should not automatically be treated as a measured probability. A useful threshold is a decision supported by evaluation, not a decorative number beside a sentence.

Preserve a test set that was not used to choose the threshold. Otherwise a rule can look dependable because it was adjusted to the same examples used to report success. Keep the exact source snapshot and expected answers, so later document changes do not rewrite the result. When a model or retrieval system changes, repeat the relevant evaluation rather than inheriting the old confidence label.

Make abstention lead somewhere useful

In the example service, a missing opening date could produce a short explanation that the provided material does not establish it. The assistant might identify the specific record needed or ask which branch the user means. It should distinguish inability to retrieve a document from evidence that the fact is unknowable. Those states require different next steps.

When sources conflict, show the conflict and their dates rather than blending them into a single confident answer. For a consequential decision, the organisation may require a person to inspect the original record. That is a proposed operating rule for this service, not a universal requirement or a claim of compliance with a particular law.

Uncertainty estimation also has costs. Generating several answers or adding separate checks can increase latency and compute use. Additional model-generated judgments can themselves be wrong. Measure whether the added step changes the errors that matter, and compare it with simpler improvements such as a better document set or clearer questions. Do not add an elaborate confidence layer before establishing that the basic evidence can be retrieved.

The practical benefit is an assistant that makes missing support visible instead of disguising it as fluency. The limitation is that no single score can remove the need to judge sources, context and consequences. Begin with a defined answerable task, include deliberate missing-evidence cases, and examine what the system does when it cannot justify an answer. Reliability includes the work of declining well, not just the ability to keep talking.