Do you not see the national security aspects of 2024 AI? It's very similar to the nuclear race where, right or wrong, there is a non-zero possibility of a winner take all supremacy scenario where the first to reach superintelligence rules the world.
We don't let private companies own nuclear weapons, yet somehow these razzle dazzle claims of OpenAI's nuclear capabilities are never followed by "so nationalize OpenAI".
No credible AI expert thinks LLMs will lead to AGI. They are hilariously bad at most things and heavily dependent on questionable data sources like reddit.
AI don't need to be competent to cause harm, just like humans. Just ask law enforcement arresting people because their face recognition AI told them to.
Not that I trust governments trying to oversee them either...
There may be a nonzero chance of developing a superintelligence (or even AGI in any meaningful sense), but it's so close to zero that it doesn't matter.
Particularly because if a superintelligence is actually created, it won't matter who did it. It won't be the servant of the creators.
How can you so confidently assert that? AI just needs to beat humans at AI research to start compounding it's growth in performance.
>It won't be the servant of the creators
Human's are very paranoid and competitive because we've had to be for tens of thousands of generations to survive. Super intelligent or not why would AI start ignoring instructions or develop a survival instinct? This line of thinking has been popular in science fiction but what is it based on? Just because something is smarter than us, it will kill us, as Eliezer Yudkowsky asserted "because it wants our atoms"? It seems like we are viewing AI through a human lens.
Superintelligence under human control is what scares the shit out of me. I see superintelligence as the only thing capable of saving humans from themselves.
Because I've seen no strong reason to think otherwise. However, everyone -- myself included -- is just guessing here. I could certainly be wrong. As could the ones of the opposite opinion.
> AI just needs to beat humans at AI research to start compounding it's growth in performance.
That's assuming that it's possible to create an AGI at all given our current direction. I seriously doubt that it is. Time will tell, though.