A firm testing or auditing a system both finds the problems and rates them, which is an odd arrangement if you think about it. I wanted to see what published reports actually offer to back up a Critical or High rating. This started out as "Do pentesters overrate their findings?" and the title changed because the sample turned out not to support that question.
I drew 20 reports at random from a public collection of 355, and coded three anonymised commercial reports separately. Of 38 findings rated Critical or High, 16 (42%) showed the attack working and nine (24%) gave no score, no reason and no demonstration at all. The split by report type is the interesting part: 29 of those 38 came from code audits, which demonstrated 7, while the application tests demonstrated all 9 of theirs. Only 3 of the 20 reports said whether they had tried to exploit what they found.
The sample isn't what its label says. The collection calls itself penetration test reports, but 16 of my 20 are source-code or smart-contract audits, where experts read code rather than attack a running system, and only 3 are application tests. Some firms do print a difficulty or likelihood beside each rating, and some lower a rating when they couldn't exploit the issue. The coding was done by AI models rather than human experts, and nothing here can show that anyone rates too high, because there's no independent re-rating to compare against.
So under half the top ratings were demonstrated, and under a quarter were bare assertions. That's not an argument that testing or auditing is worthless. My own view, which is a judgement rather than a finding, is that reports are easier to trust when they say what was tried, what worked, and why each rating was chosen.