Tailored propaganda, scams, spams, and harassment at a scale that was never seen before. Plugging metasploit into an unfiltered GPT-5 with a shell and a few proxies could be devastating. Undetectable and unstoppable bots would be available to anyone. Don't like someone? You could spend a hundred bucks to ruin their life anonymously.
Each of us could unknowingly interact with multiple LLMs everyday which would only have one purpose: manipulate us with a never-seen before success rate at a lower cost than ever.
At some point AI generated content could become more common than human content, while still being indistinguishable.
Good enough automated online propaganda could routinely start (civil)wars or genocides, Facebook already let that happen in the past, manipulating elections would become systematical even in the most democratic countries.
What already happened in those areas in the last few years, is really nothing compared to what could happen without enough regulation or barriers to entry in the next few years.
What's worse is that all of this, would not just be possible, but available to every sociopath on earth, not just the rich ones.
>> Tailored propaganda, scams, spams, and harassment at a scale that was never seen before.
I believe the state of these subjects right now is already alarming without AGI. You can't exacerbate the horror about the level of tailored propaganda and scams, etc., which you can't even foresee yourself. It isn't quantifiable.
>>Each of us could unknowingly interact with multiple LLMs everyday which would only have one purpose: manipulate us with a never-seen before success rate at a lower cost than ever.
You would build resistance pretty quickly.
>> At some point AI generated content could become more common than human content, while still being indistinguishable.
Oh, there were some numbers on that one. The number of images generated with AI is already several magnitudes larger than the number of photos humanity has produced since the invention of photography. No AGI is required either.
>> Good enough automated online propaganda could routinely start (civil)wars or genocides,
It already does, without AGI. The Black Rock guys say it's good, - war is good for business. You can squeeze the markets, make money on foreseeable deficits.
>> What's worse is that all of this, would not just be possible, but available to every sociopath on earth, not just the rich ones.
But guns available to every sociopath on earth too...
All of your arguments concern how those with malicious intent can harm us further. I would argue that Sam Altman as the sole controller of AGI is a rather unsettling prospect. If only one country possessed a nuclear weapon, that country would certainly use it against its adversaries. Oh wait, that's already a part of history...
> >>Each of us could unknowingly interact with multiple LLMs everyday which would only have one purpose: manipulate us with a never-seen before success rate at a lower cost than ever.
> You would build resistance pretty quickly.
That is adorably naive. The current thrust in LLM training is towards improving their outputs to become indistinguishable from humans, for any topic, point of view, writing style, etc.
Each of us could unknowingly interact with multiple LLMs everyday which would only have one purpose: manipulate us with a never-seen before success rate at a lower cost than ever.
At some point AI generated content could become more common than human content, while still being indistinguishable.
Good enough automated online propaganda could routinely start (civil)wars or genocides, Facebook already let that happen in the past, manipulating elections would become systematical even in the most democratic countries.
What already happened in those areas in the last few years, is really nothing compared to what could happen without enough regulation or barriers to entry in the next few years.
What's worse is that all of this, would not just be possible, but available to every sociopath on earth, not just the rich ones.