[shortoncash]

This new fear around propaganda bots seems unwarranted. I've gotten mail advertising specific political newsletters or products for as long as I've been an adult. Someone was aggregating information and targeting me long before social media was around.

I feel as if the political class is just upset that they are not protected by the usual gatekeepers and that this is a non-issue.

[api]

I think we're in the early stages of the development of what I've termed the propaganda doomsday machine. Think of it as the PR equivalent of the hydrogen bomb.

What I'm concerned about is the combination of AI and big data to generate autonomous personalized intelligent agents that can actually converse with you, or at least can tailor propaganda to your specific cognitive profile with an incredible degree of insight and accuracy. Think of it as the ability to dispatch an army of billions of con artists with each tasked to work on an individual target person and with the benefit of "fleet-wide learning." I've heard this called "AI-assisted demagoguery."

Keep in mind that it doesn't need to work on everyone. A system like that with a ~10% success rate at changing peoples' minds could allow its wielders to quite literally conquer any democratic nation. That would be enough to sway nearly every election. Non-democratic nations wouldn't be immune either as such technologies could be used to foment revolutions, though I suspect that the smarter autocrats would invest as much as it took to make sure they were the wielders and not others. You can see this in China right now where its autocrats are deploying a system very much like this against their population.

5-10% success rates could just be the beginning. As we say in information security: "attacks only get better." Iterative development and the ensuing arms race could eventually yield systems that exploit tremendous insight about the structure and function of the human mind down to the neurological level. Unfortunately unlike software we cannot patch our brains to fix vulnerabilities. Eventually it may not be possible to engage in honest public discourse at all, since any person in the public square would have a high chance of not being a person at all but an AI-powered con artist attempting to sell you something or change your mind.

I"ve also thought that this -- not Terminators shooting people with lasers -- is what an AI takeover would look like. The ultimate intelligences behind it might be AIs but are more likely to be super-wealthy and politically powerful humans wielding AI as a tool to cognitively enslave the rest of humanity to themselves.

Also what's this "the political class" stuff? If you're talking about Trump vs. Clinton, both of them are incredibly wealthy and highly connected insiders. Trump has somehow managed to pose as an outsider but 10 minutes of reading about his background and connections will disabuse you of that. He's as much an insider as Clinton, albeit in a different clique of the global superclass. The people who funded Cambridge Analytica were also members of the global superclass. What we're seeing here is not insiders assaulting the elite but different camps of elites waging propaganda wars against each other. It's the haves vs. the haves, not the haves vs. the have-nots.

[shortoncash]

Your fears are noted and valid. However, this propaganda hydrogen bomb already happened once to humanity with the advent of mass media. The wealthy were (and are) using mass media to sway the same 5-10% (and likely more) that you're referring to.

Maybe the future brings us more personalized and targeted propaganda, but the number of sources will likely be more varied. The propaganda distribution cost is lower. Moreover, I think much of humanity will adapt and counter propaganda as part of an ongoing cat-and-mouse game in the persuasion and disinformation business.

I only referred to the political class because I think this topic is only getting attention currently because of political games being played. However, I believe the current beneficiaries of the last propaganda hydrogen bomb (the current mainstream media) dislike the new market entrants and don't want to compete -- that is why they are vilifying the owners/controllers/developers of the AIs that you are also concerned about.