[hard_times]

Whatever findings they come up with, must be formalized to avoid non-determinism.

Draft an integration test of every finding.

The malicious extension makes calls to haxx0r.net? Draft a case in your integration test that intercepts this.

[port11]

But why will LLMs get this right when Web-of-Trust and blacklists didn’t? For a long time we’ve had different heuristics to detect abuse, and it’s always been a losing battle.

E.g. an extension that sends requests to an IP. Do you block all network access? IP ranges? Well, we’ve had firewalls for ages, hackers still craft successful vectors.

[hard_times]

I'd say it's different this time, because LLM can form and substantiate opinions of their own, on their own. You could actually assume you have the full source code of any binary for it to analyze, since in theory everything is decompilable, and you could then also diff between releases to save 99% tokens.

[chris_money202]

Think this is more an argument against them in security than for them. You don't want the LLM forming the opinion that something is secure when it is not because it has "microsoft" in the name of a binary or something else. There are a lot of ways to mis-direct their reasoning and it's almost impossible to control that 100%. A lot of AI providers are learning from the mistakes, so implies something is going to get hacked once in some unique way and then LLMs won't make the mistake again with 99% certainty.

[hard_times]

Let's strip all strings then before analysis? to prevent prompt injections and such.