[noduerme]

This needs to be hammered into people's understanding of the danger of LLMs at every opportunity. Enough of the general population considers things like Twitter bots to have scaled to a dangerous point of polluting the information ecosystem. The scalability and flexibility of LLMs in germinating chaos is orders of magnitude beyond anything we've yet seen.

An example I use for people is the Bernstein Bears effect. Imagine you wake up tomorrow and all your digital devices have no reference to 9/11. You ask Bing and Google and they insist you must be wrong, nothing like that ever happened. You talk to other people who remember it clearly but it seems you've lost control of reality; now imagine that type of gaslighting about "nothing happening" while the lights go out all over the world and you have some sense of what scale the larger of these systems are operating at.

[eru]

> Enough of the general population considers things like Twitter bots to have scaled to a dangerous point of polluting the information ecosystem.

It was always a good idea to ignore the cesspool that is Twitter. No matter whether we are talking about bots or lynch mobs.

Btw, I think you mean Berenstain Bears.

[rurp]

Twitter is just one example though, this problem is going to affect every single online community. If the LLM bull case is correct, the internet is going to be absolutely flooded with sophisticated misinformation.

[noduerme]

Sophisticated being key. Quantity * quality almost indiscernible from mediocre human input.

Currently we tend to understand bad information on the stream as a function where quality is linear and quantity is exponential, and individuals or human filters can still identify reject the lower 99% as spam. Every point closer on the graph the quality comes to resemble human-made content represents an exponential degree of further confusion as to base facts. This isn't even considering whether AI develops its own will to conduct confusion ops; as a tool for bad actors it's already there, but that says nothing of the scale it could operate at eventually.

The sophistication of the misinformation is exactly the point: That's the mass multiplier, not the volume.

[edit] an interesting case could be made that the general demand for opinionated information and the individual capacity to imbibe and adjudicate the factuality of the input was overrun some years ago already... and that all endeavors at misinformation since then have been fighting for shares of an information space that was already essentially capped by the attention-demand. In that paradigm, all social networks have fought a zero-sum game, and LLMs are just a new weapon for market share in an inflationary environment where all information propagated is less valuable as the volume increases and consumption remains static. But I think this is the least worrisome of their abilities.

[deafpolygon]

_Berenstein_ Bears. Don't fall for the fake news. ;)

[biztos]

I’d never heard of the effect, but fell for it anyway because “-stain” is so unusual.

https://en.wikipedia.org/wiki/Berenstain_Bears#Name_confusio...

[brutus1213]

To save people the search: https://www.youtube.com/watch?v=7h8jqEs_8kA

[erosenbe0]

Would univeral adoption of digital signatures issued by trusted authorities alleviate this problem to any degree?

For example, my phone would automatically sign this post with my signature. If I programmed a bot, I could sign as myself or as a bot, but not as another registered human. So you'd know the post came from me or a bot I've authorized. Theft or fraud with digital signatures would be criminalized, it isn't already.

[noduerme]

No, I think we should check for an actual pulse before people post.

Your comment is wild, by the way. You think people should be allowed to run a bot farm, as long as they can digitally sign for it... but people who don't pay for a signature should be arrested?

[erosenbe0]

I'm just asking if some system of using digital signatures could help weed through the inevitable proliferation of bots and deepfakes and ai agents.

I'm pretty sure it's already illegal to steal someone else's signature in some jurisdictions.

There would be no legal requirement to use a signature. No change there. Just as you cam send postal mail today with a return address and no name, and you can buy items with paper cash, and so forth. The government would give out verified signature, or the phone providers, and it'd be free. I don't really have the answers.