|
|
|
|
|
|
by EMM_386
37 days ago
|
|
|
If an AI agent finds zero bugs in a software utility, how can that be viewed in the sense the AI agent is not very good at finding bugs? What if there are actually zero bugs? > Five issues felt like nothing as we had expected an extensive list. The expectation here may not match reality, but not necessarily because Mythos isn't as capable as claimed. curl may just happen to be a well-hardened tool that doesn't have too many security vulnerabilities in its present state. |
|
|
> More to find
> These were absolutely not the last bugs to find or report. Just while I was writing the drafts for this blog post we have received more reports from security researchers about suspected problems. The AI tools will improve further and the researchers can find new and different ways to prompt the existing AIs to make them find more.
> We have not reached the end of this yet.
> I hope we can keep getting more curl scans done with Mythos and other AIs, over and over until they truly stop finding new problems.
And that makes sense, it'd be quite the argument of coincidence to say there was just 1 proper find remaining & it was only Mythos that managed to find it just at the point in time it released while the other projects have been hoovering up every other find quickly until that point. Possible, but not the safest assumption to start questioning with.