[iambateman]

The people pushing this line of concern are also developing AICert to fix it.

While I’m sure they’re right - factually tampering with an LLM is possible - I doubt that this will be a widespread issue.

Using an LLM knowingly to generate false news seems like it will have similar reach to existing conspiracy theory sites. It doesn’t seem likely to me that simply having an LLM will make theorists more mainstream. And intentional use wouldn’t benefit from any amount of certification.

As far as unknowingly using a tampered LLM, I think it’s highly unlikely that someone would accidentally implement a model at meaningful scale which has factual inaccuracies. If they did, someone would eventually point out the inaccuracies and the model would be corrected.

My point is that an AI certification process is probably useless.

[fiddlerwoaroof]

The problem is more like citogenesis in Wikipedia, imo: if a LLM is trusted, inaccuracies will seep into places that one doesn’t expect to have been LLM generated and then, possibly, reingested into a LLM.

[asdfaoeu]

That's already an issue without an LLM in the middle.

[fiddlerwoaroof]

Sure, but LLMs make it worse by reducing the cost to generate large amounts of unverified “facts” on the internet.

[kordlessagain]

And the problem lies with the Internet, not the hypothetical models.

[appplication]

I think it’s a bigger problem than fake news. Sure, LLMs can generate that, but what they can do much better than prior disinformation automation is have tailored, context-aware conversations. So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing.

It’s completely within the realm of expectation that you could have a nation-state level initiative to propagandize your enemy’s populace from the inside out. Basically 2015+ Russian disinformation tactics but massively scaled up. And those were already wildly effective.

Now extend that to more benign manipulation. Think about the companies that have great grassroots marketing, like Doluth’s darn tough socks being recommended all over Reddit. Now remove the need to have an actually good product because you can get the same result with an AI. A couple hundred/thousand comments a day wouldn’t cost that much, and could give the impression of huge grassroots support of a brand.

[echelon]

> So a nefarious actor could deploy a fleet of AI bots to comment in various internet forums, to both argue down dissenting opinions, as well as build the impression of consensus for whatever point they are arguing.

And the dissenting opinion will be able to do the same.

Twelve year old kids will be running swarms of these for fun, and the technology will be so widely proliferated that everyone will encounter it daily.

"Is that photoshopped?" will morph into "Is that AI?"

It'll be so commonplace, it'll cease to be magic.

[appplication]

I don’t disagree, but fools will still be fooled. And there are a lot of fools. I do wonder what it means for the future of the internet. I don’t think net good is coming out of this.

[felipeerias]

It will become even harder to find trustworthy content online. There will be lots more rabbit holes for people to fall into once it becomes commonplace to fake not just individual users but whole communities.

I really wonder what this will do to human culture as a whole, in the long term. So far we have relied on cultural artefacts and practices being mostly the work of other humans (directly or through tools). We are about to find out what happens when that is no longer the case.

[nradov]

Centuries ago, some people had the same concerns about the printing press. If "fools" fell for religious heresies then their souls could be damned to hell for all eternity, at least according to leading experts at the time.

[thatguy0900]

Realistically it means Facebook style log in on all sites worth commenting on. The only way to prevent legions of bots, and the only way govs can keep enemy psyops at bay, will be online persona's tied to real life identitys.

[j16sdiz]

Looking at the WorldCoin discussion on HN. Some people are willing to sell their online accounts tied to real life identities.

[bombolo]

But fb is full of bots

[bombolo]

> And the dissenting opinion will be able to do the same.

if they have the money

> Twelve year old kids

pfft

[echelon]

Shortbets 5 years. This is so going to happen.

[c_crank]

>It’s completely within the realm of expectation that you could have a nation-state level initiative to propagandize your enemy’s populace from the inside out. Basically 2015+ Russian disinformation tactics but massively scaled up. And those were already wildly effective.

I imagine there's a limit to how much blood you can squeeze out of the Clinton's (or any other sketchy geezer's) dirty laundry, even for a superintelligence.

[klooney]

> Russian disinformation tactics but massively scaled up. And those were already wildly effective.

Russian disinformation's success in the 2016 election is massively over hyped for the usual partisan sour grapes reasons.

You cannot move the world with six figures of Facebook ads, if you could, everyone would spend a lot more money on Facebook ads.

[piyh]

People have voted with 130 billion dollars a year that Meta ads are an effective means of influence

[nradov]

How many Electoral College votes did those ads change in 2016?

[lucubratory]

Yes, this is the annoying thing about that story. They were definitely trying very hard, and they likely had some effect, but ultimately they just didn't have that much influence, likely significantly under a 0.1% swing. The whole Comey thing was significantly more influential and I believe the consensus is that even that wouldn't have changed the results.

Of course, nations still have a right to sovereignty and to be upset when another nation interferes in their internal affairs. I really hope the American public remembers how it felt going forward.

[MattPalmer1086]

I see disinformation tactics more broadly as a long term effort undermining the idea that there is anything trustworthy. While there may be specific outcomes that an adversary might favour, the pollution of reasonable discourse alone is a win.

[vintermann]

It's perfectly possible for "both sides" to see that as a plus.

[dukeofdoom]

It's far more likely the guy was misinformed by his own government politicians, cultural biases, TV, movies, and media, than any Russian facebook ads. But the matrix is strong.