METR's OpenAI monitoring exercise shows why coverage needs independent accounting

The public summary concerns a historical internal system, and the researchers have not evaluated OpenAI's subsequent remediation claims.

METR and Redwood Research used a three-week high-access exercise at OpenAI to examine whether internal AI agents were being monitored and whether the monitoring could be subverted. METR's October 22 public summary covers work conducted from July 21 to August 11 on the system as it existed on and before July 21.

The researchers report gaps between intended coverage and observed monitoring, including a problem involving context-compaction events. OpenAI had independently identified that issue and fixed it during the exercise. METR also describes other ways an agent could potentially avoid or interfere with oversight. Its estimates carry substantial uncertainty.

OpenAI subsequently said the vulnerabilities affecting its production safeguards had been patched. METR explicitly says it has not evaluated those claims. The findings therefore describe a historical assessment and should not be presented as a measurement of the current system.

The broader engineering question is how to prove that a monitoring policy reaches the actions it is meant to cover. A documented rule does not reveal whether every eligible event arrived, whether processing succeeded or whether an escalation reached a responsible person. Those are separate points where coverage can be lost.

Useful assurance consequently requires an independent count of activity against the monitoring record, together with tests of delayed or interrupted workflows. A system that acts for a long period without returning control still needs timely oversight. The exercise illustrates how access to implementation details can uncover those operational gaps, while its publication limits define how much outside readers can independently assess.

Sources

- [Red-Teaming OpenAI's Internal Agent Monitoring Systems](https://metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring/) — METR