Reliability lags capability
24 February 2026 (AI as Normal Technology). From the draft paper Towards a Science of AI Agent Reliability by Stephan Rabanser, Sayash Kapoor, Arvind Narayanan and colleagues, measuring AI agent reliability across 14 models, summarized on AI as Normal Technology, February 2026.
New Paper: Towards a science of AI agent reliability
AI and safety · AI and science · 2026
Who else?
What it refers to: The finding that agents based on 14 models from OpenAI, Google, and Anthropic improved substantially in accuracy over 18 months but only modestly in reliability.
Other voices
- “Test how reliably they follow task restrictions before granting them broader access or autonomy” Irregular,
- “Hallucination is not a solved problem” Sundar Pichai, Builder,
- “These systems are brittle” Radford et al.,
- “Capability improvement trends are rapid, and GPT-5 is somewhat above-trend” Beth Barnes,
- “You have no business running all the world's software” Gary Marcus, Researcher,
- “This dynamic is called recursive self-improvement, and it is starting to happen” Dario Amodei, Builder,










