AI incident counts are signals, not a universal risk rate

The 2026 international safety report and OECD monitoring methods show why evidence needs denominators, context and uncertainty.

A growing list of AI incidents can tell us that problems deserve investigation. It cannot by itself tell us how often every AI system fails. The count depends on what is reported, which events are included and how widely the technology is used. Public discussion improves when those distinctions stay visible, especially when incident evidence informs consequential policy or purchasing decisions.

The [International AI Safety Report 2026](https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026), published on 3 February, synthesises research on general-purpose AI capabilities and risks. It does not endorse a particular policy approach. The OECD’s [incident-monitor methodology](https://oecd.ai/en/incidents-methodology) describes a different evidence source, while its [2025 reporting-framework paper](https://www.oecd.org/content/dam/oecd/en/publications/reports/2025/02/towards-a-common-reporting-framework-for-ai-incidents_8c488fdb/f326d4ac-en.pdf) addresses common reporting structure. A research synthesis and a monitored collection of reports answer different questions.

An original illustrative public dashboard records fifty reports involving an AI customer assistant. That number is concerning, but interpretation requires context. Were there fifty unique events or multiple stories about the same one? How many interactions occurred? Were the reports confirmed? Did the system change during the period? Without those details, a precise failure percentage would be invented.

The absence of reports is also ambiguous. An organisation may have effective controls, or users may lack a reporting route. Staff may classify problems under a general service category without recording the AI component. A quiet dashboard is therefore not sufficient evidence of safety. Monitoring needs a deliberate process for finding and reviewing failures.

A practical incident record can distinguish what a person reported from what investigators confirmed. It can identify the task, the observed consequence, the system version and the resolution status. Keep private details restricted and use safe descriptions in public summaries. A report should remain visibly unresolved until the relevant facts are established.

Near misses are useful too. A reviewer may catch an incorrect deadline before it reaches the public. That event reveals a model weakness and a working control. Recording only completed harm would miss both. The organisation can learn whether similar errors recur and whether the review process is keeping pace with the workload.

Denominators need to match the question. If the team wants to estimate incorrect answers per interaction, it needs representative interaction sampling and a defined error criterion. If it wants to understand severe harm, a random sample may miss rare consequential events and needs complementary investigation. One measure cannot carry every governance decision.

Changes in reporting can alter the apparent trend. A new feedback button, media attention or a better internal taxonomy may increase recorded incidents even if the underlying service improves. Explain those changes beside the chart. Otherwise readers may infer deterioration from more complete measurement, or infer improvement from a reporting process that stopped working.

A research synthesis provides broader context but still has limits. Studies use different tasks, systems and definitions. Evidence can be strong in one area and uncertain in another. The international report’s role is to assemble and assess that literature. A team applying it should identify the part relevant to its service rather than using the report’s existence as proof of its preferred conclusion.

The benefit of common reporting is more comparable, reusable evidence. The failure is a dramatic count stripped of definitions or treated as a universal risk rate. Policy teams can use incident reports to identify failure patterns, prioritise inquiry and improve controls. They should describe the reporting boundary and avoid numerical precision the data cannot support.

For an organisation, the practical task is to make reporting usable and investigation traceable. Provide an accessible way to report a problem, preserve the relevant configuration and publish appropriate corrections. Combine incident review with representative testing and service measures. That produces a more faithful account than either a success-only benchmark or an unqualified incident total.

An illustrative monthly review can select one resolved event, one near miss and one unresolved report. Ask whether the evidence supports each classification and whether the response changed the service. This keeps the discussion connected to concrete facts and highlights uncertainty that a single dashboard number would conceal.