An AI evaluation should say which question it did not answer

Current UK assurance pages and dated NIST resources help make a test’s excluded questions as visible as its successful cases.

An answer can be correct in a test and still be unsuitable for the task a person believes the system performs. An evaluator may measure document retrieval while a user expects policy interpretation. A team may test familiar wording while staff later ask questions in another language. These are not minor reporting details. They define the boundary between the evidence collected and the decision someone wants to make from it.

For defence administration, that boundary is particularly important because a general-purpose assistant can appear to offer a broader competence than its approved task. A public educational discussion need not explore operational uses to make the point. A reference service, training-document search or administrative drafting tool already raises questions about version, context, uncertainty and human review. The evaluation should describe those questions separately instead of allowing one aggregate result to stand for all of them.

Guidance provides structure, not a passed test

The UK Defence AI Practitioner’s Handbook page [The importance of AI assurance in Defence](https://www.digital.mod.uk/policy-rules-standards-and-guidance/aiph/the-importance-of-ai-assurance-in-defence), published and updated on 27 February 2026, describes assurance as evidence organized into a logical argument. It emphasizes the system’s use and the interaction between people and technology. That is guidance on how to reason about evidence, not a finding that a particular system is dependable.

The separate [DAIC AI Assurance Framework page](https://www.digital.mod.uk/policy-rules-standards-and-guidance/aiph/the-defence-ai-centre-ai-assurance-framework), with the same displayed date, organizes its explanation around people, process, evidence and technology. Its account is explicitly connected to applying UK defence policy. A reader should not mistake those categories for a universal certification scheme or treat completing a template as the same thing as observing the relevant behaviour.

NIST’s [AI Risk Management Framework page](https://www.nist.gov/itl/ai-risk-management-framework) dates the original framework’s release to 26 January 2023 and describes voluntary use. The living page also identifies revision work. This article refers to that dated framework and its stated scope, rather than claiming it is a newly released defence standard. A general framework can help organize questions while the actual application still needs an appropriate, context-specific assessment.

The [Generative AI Profile publication record](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) separately dates that cross-sectoral companion resource to 26 July 2024; its web record shows an 8 April 2026 update. Publication and page maintenance are different events. Neither date proves that a particular assistant was tested against the resource or that a revised interface retains earlier evidence.

A fictional public-manual assistant

Imagine a fictional adult team in Bristol testing an assistant for locating approved public training documents. It uses synthetic questions and no classified, personal or operational material. The exercise does not model targeting, weapons or field decisions. The first task is modest: find a named document edition and identify the passage that supports an administrative answer.

Before testing, the team separates possible questions. Can the system retrieve the right document? Does its answer accurately reflect the passage? Can the reader identify uncertainty or a missing source? Does the interface distinguish an old edition from a current one? A result for retrieval should not silently become a result for interpretation, and a result for either should not become a claim about the user’s final decision.

The team writes an evaluation boundary for each question. It names the document set, languages, permitted tasks and excluded decisions. It records the versions of the assistant and collection. That record is not bureaucracy added after testing; it makes the test interpretable. Without it, a later reviewer cannot tell whether an unexpected answer reflects a new document, an interface change or a condition that was never included.

Build comparisons that preserve uncertainty

A worked exercise includes questions with adequate support, questions that require a missing edition and questions where two public documents describe different periods. The team defines those categories before examining the assistant’s answers. A refusal to answer a deliberately unsupported question can be useful behaviour. It should not be counted as an ordinary failure merely because the demonstration was designed to produce fluent responses.

Reviewers then inspect the supporting passage rather than agreeing only on whether an answer sounds plausible. They preserve disagreement where the reference material is ambiguous. If the expected answer depends on an unstated interpretation, the evaluation needs correction too. Calling the reference set ground truth does not exempt it from review. The test record should explain how disputed cases were handled and which remained unresolved.

The human comparison matters. An experienced librarian, a new staff member and a person unfamiliar with the document vocabulary may interact differently with the same screen. The team can observe whether each person notices the edition and source link, without making a general claim about all users. These observations concern the complete task and interface. A model-only score cannot supply them automatically.

A change should inherit only relevant evidence

Suppose the fictional team adds another language to the collection. Its earlier English-document retrieval result remains a historical observation. It does not establish equivalent behaviour for the new material. The team should identify which questions have changed and which comparisons are still relevant. Retaining the earlier boundary is more useful than replacing the old report with a broader description of the service.

The same applies when an answer cannot be confirmed after an interruption. The test should distinguish an unavailable response, an unsupported answer and a completed answer the user could not inspect. These states affect recovery differently. A safe human message can explain what remains uncertain while preserving the question and any notes. It need not expose raw technical diagnostics to make the failure visible.

Report the unanswered questions

An evaluation report should make its exclusions easy to find. The fictional service might have evidence for locating passages in a defined public collection, while lacking evidence for legal interpretation, new languages or changed document formats. Saying so does not negate the useful result. It tells the next decision-maker what the result can carry and what requires another investigation.

Potential benefits include clearer comparisons and fewer misunderstandings about an assistant’s approved role. No accuracy rate, cost reduction or safety improvement is measured in this fictional exercise. The official guidance supplies concepts and responsibilities; the proposed trial supplies a way to make its own limits inspectable. An honest evaluation is strengthened by a clear account of the questions it has not answered.

Evidence diagram and readable guide


Original explanatory diagram: practice cases and human-task observations provide separate evidence for a fictional scoped evaluation.

[Open the complete evidence diagram](/assets/defence-workflow-evaluation-boundaries-20261015.svg)

Read the diagram

**An AI evaluation should say which question it did not answer**

A proposed test for a fictional public-reference service. Guidance supplies structure, not a passed evaluation or certification.

- **Bounded public collection** Record versions, languages and excluded decisions.

- **Evaluation questions** Separate retrieval, reflection of a passage and user review.

- **Held-out practice cases** Include unsupported and conflicting material.

- **Human task observations** Observe whether adults can inspect the source and edition.

- **Scoped result** Report only what the comparison established.

- **Unanswered questions** Preserve limits that need a different investigation.

Case results and human-task observations answer different questions; neither substitutes for the other.

How the connections work

- The bounded public collection defines which evaluation questions can be asked.

- Evaluation questions determine the held-out practice cases.

- Evaluation questions also determine which human-task observations are needed.

- Practice-case evidence contributes to the scoped result.

- Human-task observations contribute separate evidence to the scoped result.

- The scoped result explicitly identifies questions it did not answer.

Relationship key

Practice-case evidence and human-task observations reach the scoped result through separate paths. The final record carries unanswered questions forward; it does not certify broader uses.