[cyanydeez]

Does no one defending mythos understand how nested foreloops work?

loop through each repo: loop through each file: opencode command /find_wraparoundvulnerability next file next repo

I can run this on my local LLM and sure, I gotta wait some time for it to complete, but I see zero distinguishing facts here.

[johnfn]

No one is saying your nested for loop idea because it won't actually work in practice. In short, the signal to noise ratio will be too high - you will need to comb through a ton of false positives in order to find anything valuable, at which point it stops looking like "automated security research" and it starts looking like "normal security research".

If you don't believe me, you should try it yourself, it's only a couple of dollars. Hey, maybe you're right, and you can prove us all wrong. But I'd bet you on great odds that you're not.

[Dylan16807]

The question is how customized those hints were. That changes whether looping over an entire code base is possible or not.

[fulafel]

Aisle said they pointed it at the function, not the file. So, the nr of LLM turns would be something like nr of functions * nr of possible hints * nr of repos.

Could indeed be a useful exercise to benchmark the cost.

This would still be more limied, since many vulnerabilities are apparent only when you consider more context than one function to discover the vulnerability. I think there were those kinds of vulnerabilities in the published materials. So maybe the Aisle case is also picking the low hanging fruit in this respect.

[u_fucking_dork]

Please do so, looking forward to your write up

[yorwba]

When people criticize Aisle's methodology, they aren't "defending Mythos," they're bashing Aisle for their disingenuous claims.