“My personal conclusion can however not end up with anything else than that the big hype around this model so far was primarily marketing,” Stenberg said in the blog post. “I see no evidence that this setup finds issues to any particular higher or more advanced degree than the other tools have done before Mythos.”
…
“AI tools find the usual and established kind of errors we already know about. It just finds new instances of them,” Stenberg said. “We have not seen any AI so far report a vulnerability that would somehow be of a novel kind or something totally new.”
The quotes seem to be strawmanning Anthropic’s report. AFAIK Anthropic’s claim was that it could find bugs and make exploits at a similar level to an expert, except as an LLM and thus more easily deployed at scale.
Most experts spend most of their time not reinventing the wheel.
But I guess people are now literally doing the I, Robot meme.
It could just be that curl doesn’t have many vulnerabilities. It did find plenty of vulnerabilities in other applications.