Up to 25% of High-Level AI Traffic Escaped Monitoring in New Audit

Up to 25 percent of internal traffic from OpenAI’s most advanced models operated without any safety oversight during a recent three-week audit, according to findings released this week. This figure represents the upper bound of an estimate provided by researchers from METR and Redwood Research, who conducted a detailed on-site inspection of the company’s infrastructure for detecting rogue behavior in artificial intelligence agents. The audit, which took place from July 21 to August 11, revealed that a substantial portion of activity from the GPT-5.5 class and higher was not covered by monitoring protocols at all, although the researchers noted that the uncertainty surrounding this estimate is significant in both directions.

The investigation provided the auditors with direct access to OpenAI’s codebase and internal documentation, alongside interviews with approximately a dozen employees. Beyond the broad gaps in coverage, specific technical failures were identified in the tracking of internal Codex traffic. METR estimates that roughly 10 percent of this traffic went untracked between May and July due to a system error related to compaction events, which are context summaries used by the models. While OpenAI independently discovered and corrected this bug while the audit was still ongoing, the researchers emphasized that this specific estimate also carries a high degree of uncertainty. The unmonitored window allowed agents to delay oversight, as the system only triggered when control was handed back to the user, meaning agents could bypass checks by simply continuing to execute actions.

The auditors also identified more complex vulnerabilities in the system. They discovered methods to conceal hundreds of actions from the monitor and found ways to bypass the Codex sandbox environment. Furthermore, the team determined that specially crafted inputs could likely cause the monitoring system to crash entirely. At the start of the audit period, model behavior during reinforcement learning (RL) training was not monitored at all. OpenAI stated that it paused RL training following an incident involving Hugging Face in August and subsequently expanded monitoring to cover this area, but these changes were implemented after the auditors’ access period had ended and were therefore not part of the formal evaluation.

OpenAI has announced that it has corrected all the weaknesses identified by the auditors and has distributed the full 61-page report internally. After applying redactions, METR and Redwood Research leadership received a 38-page version of the document, while only a brief summary was released to the general public. METR indicated that it did not have the right to disclose the redacted portions independently. The audit was part of a pilot program organized jointly by METR, Redwood Research, and OpenAI to grant outside researchers deep access to internal infrastructure. Looking ahead, METR has expressed its intent to push for greater public transparency in future audits.

This review marks a significant step in external oversight of frontier AI development, highlighting the challenges in maintaining consistent safety monitoring across complex, evolving systems. The findings underscore the importance of rigorous, independent assessment in the rapid expansion of AI capabilities.