Desk study30 minutes

Do OWASP's LLM and Agentic Top 10s Describe Observed Failures?

A test of whether the lists' categories map onto the failures that have actually been documented in public

OWASP publishes two widely used lists of AI security risks, one for applications built on large language models and one for agents. I wanted to know something narrow: whether real incidents actually fit into one category on these lists. Not whether the lists are useful, which is a different question.

I used OWASP's own incident tables, a peer-reviewed study of 100 security advisories, a catalogue of 487 agent incidents, and 36 incidents I pulled from primary sources and had coded separately by two Claude models. In most of that evidence, fewer than half of incidents fit a single category. Agent attacks usually span several, because the lists split one attack into stages: how the instructions got in, what leaked, how it got out. Set aside incidents that fall outside the lists' stated scope and roughly half fit one category. Some do fit cleanly, like chatbot misinformation, customers tricking chatbots, and malicious model files.

Two common things sit outside both lists. Using AI as a weapon or a fraud tool is frequent in OWASP's own incident data, and OWASP considered it as a category and chose not to publish it. Ordinary software flaws in AI products are also common and deliberately excluded. It's worth knowing that OWASP's 2026 LLM ranking is three-quarters practitioner vote and one quarter incident data, and OWASP's own analysis found the two agreed only weakly.

So the lists describe some of what happens, but not the shape of most of it. Another taxonomy classified every incident in its sample, so complete coverage is clearly possible, which makes this a choice rather than a limit of the problem. I'd use the lists for shared vocabulary and not expect a real incident to land in one box.

The full paper

The full paper is about a 30-minute read. It has the claims I checked, what I used, what I rejected and why, and dated copies of every source with hashes, so there is a record of exactly what I was working from.

Ask if you want to read it.