Legal AI citations need two checks: existence and relevance

Retrieval can reduce some errors without making research infallible. A reliable workflow checks the authority and the proposition it is meant to support.

A legal citation can be genuine and still fail to support the sentence beside it. AI research therefore needs two checks: whether the authority exists and whether it actually establishes the proposition claimed. A tool that retrieves documents may improve the first step while leaving the second uncertain. The reader must still examine jurisdiction, date, context and the status of the authority.

A [2024 study of specialist legal research tools](https://arxiv.org/abs/2405.20362) found errors despite retrieval-based approaches. A [separate study of general models](https://arxiv.org/abs/2401.01301) examined legal hallucinations in defined tasks and model versions. The Solicitors Regulation Authority’s [August 2026 warning](https://www.sra.org.uk/solicitors/guidance/misuse-ai) addresses inaccurate information and client confidentiality in regulated legal work. These sources support verification; the studies do not establish a current error rate for every product or every legal question.

This article is educational reporting, not legal advice. Lawyers must consider the rules and professional obligations applicable to their own jurisdiction and matter. Researchers and clients should not treat an AI answer as a substitute for qualified review. A historical benchmark is useful evidence about a tested configuration, not a permanent ranking of vendors.

An original illustrative research task asks an assistant to identify authorities on a procedural deadline. The assistant returns three cases and a confident conclusion. The reviewer first locates each case in an authoritative source and confirms its citation. Then the reviewer reads the relevant passage, checks the court and date, and examines whether later developments affect the proposition. The task remains incomplete if any of those steps fail.

False premises are particularly revealing. A question may assume that a court adopted a rule it never adopted. A system trying to be helpful can construct a plausible explanation rather than challenge the premise. The evaluation can include prepared questions with missing or incorrect assumptions. A useful response identifies the gap and asks for clarification or explains that the cited material does not support it.

Retrieval is not a guarantee because the retrieved text can be irrelevant, incomplete or misunderstood. An answer may combine language from several sources into a proposition none of them makes. It may omit a qualification that changes the result. Showing links improves inspectability, but the reviewer needs enough surrounding text to decide whether the link supports the actual claim.

A firm evaluating a product can prepare a set of resolved, de-identified questions. Include different jurisdictions, older and newer authorities, disputed points and questions for which the correct answer is uncertainty. Keep part of the set away from prompt development. Reviewers should label the expected sources before seeing the tool’s answers, so a polished output does not bias their assessment.

Measure more than citation count. Record nonexistent authorities, wrong jurisdiction, unsupported propositions, omitted qualifications and reviewer time. A system that supplies many real cases may still slow research if staff must untangle irrelevant material. Conversely, a tool that quickly identifies useful passages can be valuable even when it requires careful final analysis.

Confidentiality belongs in the same workflow. A research prompt can reveal client circumstances, strategy or a sensitive chronology. Use only approved tools and the information permitted under the firm’s policies and applicable obligations. Evaluation datasets should avoid private matter details. A public demonstration is not a suitable place to test a real client’s unresolved question.

Version changes require renewed evidence. A provider can update the model, retrieval method or document collection. Keep the tested product configuration and date with the results. Ask whether the tool explains what changed and whether prior evaluation cases can be rerun. Historical research should be cited accurately without assuming that an old result automatically transfers to a later release.

The benefit is faster discovery of material that professionals can verify and use. The failure is a persuasive answer that survives review because its citations look credible. Two deliberate checks help prevent that: establish the authority, then establish the proposition. The final legal judgment remains with the responsible professional, and the evidence record should show how the conclusion was reached.