Hacker News new | ask | show | jobs
by FranklinJabar 132 days ago
Is there any reason to think that intelligence (or computation) is the thing preventing these fears from coming true today and not, say, economics or politics? I think we greatly overestimate the possible value/utility of AGI to begin with
1 comments
johnfn 132 days ago
I mean, sure, but I don't want to give my aggressive enemies a bunch of weapons to use against me if I don't have to - even if that's not the primary thing I am concerned about.
link
FranklinJabar 132 days ago
Right but how would a chatbot be considered a weapon? Unless you're engaged in an astroturfing war on reddit it doesn't seem very useful.

Most forms of power are more proportional to how much capital you control than anything related to intelligence.

link
johnfn 132 days ago
Consider that an iPhone zero-day could be used to blackmail state officials or exfiltrate government secrets. This isn't even hypothetical; Pegasus[1] exists, and an iPhone zero-day was used to blackmail Jeff Bezos[2]. This was funded by NSO group. Opus is already digging up security vulnerabilities[3] - imagine if those guys had 1000x instances of Claude Code to search for iPhone zero days 24/7. I think we can both agree that wouldn't be good.

[1]: https://en.wikipedia.org/wiki/Pegasus_(spyware) [2]: https://medium.com/@jeffreypbezos/no-thank-you-mr-pecker-146... [3]: https://news.ycombinator.com/item?id=46902909

link
Cyphase 131 days ago
> Opus is already digging up security vulnerabilities[3] - imagine if those guys had 1000x instances of Claude Code to search for iPhone zero days 24/7. I think we can both agree that wouldn't be good.

If an LLM can be used to do that and find things (and they already have been), Apple (and everyone else) will run their code through it before releasing it. Sure, there'll be a transition period with existing code and while the tech is unevenly distributed. But in the hunt for potential zero days, developers can check their code before people are using it.

link
johnfn 127 days ago
It seems easy enough to say this, but I think reality is more complex. What about all of Apple's library dependencies? What about the dependencies of those dependencies? What about the Linux kernel? What about openssl? What about..?
link
Cyphase 126 days ago
What about them? All of those things could be checked as well.

The premise was that bad actors could use Claude Code and other available tools to find zero days. If such tools are available, good actors can use them, too, and they can use them before code is deployed. After a transition period, all existing code will have been checked.

There may be a long tail due to a large surface area of prompting techniques, but the better the tools get, the more advantage to good actors; as long as the good actors have equal or better access to the best tools, of course.

But I agree, reality is more complex.

link