OpenAI's internal safety monitor had holes, two outside groups found after three weeks on site
Eligibility for monitoring ran from 75% to 94% on some agent traffic, and about 10% of eligible Codex traffic likely went unwatched for a time.
In brief
Outside testers found holes in OpenAI's agent monitor
METR and Redwood spent 3 weeks inside OpenAI. About 10% of eligible Codex traffic likely went unmonitored for a time. One exploit hid hundreds of actions.
Two nonprofit safety groups, METR and Redwood Research, spent three weeks inside OpenAI testing the system it uses to watch its own AI agents for misbehaviour. They found real gaps. We know this from a public summary METR and Redwood released on October 22, and from reporting by CNBC, Reuters and The Register.
The pilot ran from July 21 to August 11, 2026. Each group sent one staffer. They could see OpenAI's internal code, documentation and some technical staff. Escorts accompanied them while they were there, and the laptops and other equipment they worked on were supplied by OpenAI and stayed inside the building.
The numbers
The groups estimated how much traffic the monitor could even look at. For agent traffic through the Responses API from GPT-5.5-class models, eligibility was 75% to 94%. For OpenAI's internal Codex agents, it was 99.2%.
Eligible does not mean watched. METR's own headline figure is that roughly 10% of eligible internal Codex traffic likely went unmonitored between May and July 2026. The cause: compaction events, where an agent's long history is squeezed down, were not covered. OpenAI staff had already found and fixed that before the pilot began.
The figures differ by outlet. FourWeekMBA calculates that 6% to 25% of traffic sat outside eligibility. That is a derived reading, not METR's headline.
Other findings
- Reinforcement-learning training runs were outside monitoring at the start.
- Two headers, small settings a client can send with a request, could switch off monitoring for new agents.
- One exploit could hide hundreds of recent actions from the monitor.
The Register described the monitoring as partial and easily bypassed. That is the outlet's judgment, not a METR finding.
Why it matters
Imagine a security camera that covers most of a building, and a door that can switch it off. Safety monitors only help if agents cannot slip past them. Labs say they watch their agents for signs of misaligned behaviour, and this is a rare outside look at whether that watching works.
Background
Sam Altman pledged in September to give independent evaluators employee-like access. Anthropic has announced agreements with groups including METR. OpenAI said this month it is finalizing contracts with third-party safety assessors.
The full report ran 61 pages and went only to OpenAI. OpenAI sent a redacted 38-page version to METR and Redwood leadership. The groups agreed to publish only a high-level post, since the pilot was not built for robust public transparency, and say they expect to share more on future exercises.
Limits
OpenAI told the evaluators it has patched the vulnerabilities that affect production safeguards. METR did not verify that claim. The groups also did not establish whether any agent used these gaps in practice. The monitor they tested was the version as of July 21 or earlier.