[Terr_]

For a dash of dystopia: Imagine a company starts using that same LLM fuzzy-matching [0] against what you intend to be normal search activity, to detect "bad" queries and "bad" users who will be blocked. Maybe they'll delete your SSO/email/videos/photos too, who knows.

I can easily imagine it happening, especially after some point where they start using the same systems to "enhance" your query.

[0] To be specific, your searches will be placed into a narrative document template, where a character Mr. Safety Bot is about to speak a verdict, and then the LLM story-generator decides whether it "fits" for Mr. Safety Bot to declare you banned.

[userbinator]

Orwell's idea of "wrongthink" is more relevant than ever.

[Terr_]

Much like with 1984, I think the promising/terrifying part is that it's not an issue with technology per se, but about a society that has somehow decided to allow Terrible Things to happen. (Often via too much power in too few hands with no accountability or alternatives.)

For example, imagine that there are 20 great search-engines around the world (who don't collude), and it hits rather differently.

[Cthulhu_]

I think (armchair analysis) it's a combination of two things, one is that it's by choice - people enjoy using LLMs, they find it valuable and convenient. As for social media, companies and society itself successfully managed to gamify people sharing their personal lives on the internet, to voluntarily reduce their own privacy. Also because they still believe they have a choice in the matter, by e.g. not posting or sharing things.

The other thing is the boiling frog analogy, it wasn't a sudden "we're at war now so you get a camera in your house" moment (iirc 1984 skips over the transition to an authoritative state though), it's a slow, gradual progress. People got used to taking selfies, then applying a filter, then using facial scans for identification, then a cool app that puts your face on movie scenes and now the company behind faceapp and co has detailed facial scans of millions of people.

Europe tried to limit it via legislation, but that's a lot of after-the-fact policing and that's just Europe.

[pona-a]

Orwell also imagined a future where Speakwrites made everyone unable to write by hand, making their thoughts dull and tanged as a long unprepared monologue tends to be.

I don't think he's anywhere on your side.

[DaSHacka]

That sounds like AI though, no? I've seen many below-average writers that are now dependent on it to write even medium length passages

[SoftTalker]

Are they dependent on it, or just prefer it because they don't like writing?

[butlike]

What does it matter? They're not writing on their own. All roads lead to Rome with this one

[pona-a]

To make the point explicit, Orwell wouldn't find most dystopian the fact your Speakwrite refuses to write anti-state messages, and support the right to subsidized local uncensored Speakwrites. The battle for the public minds was lost when they forgot how to write.

[brookst]

Orwell imagined it 80 years ago, not sure it’s more relevant just because someone else imagines it today.

[genxy]

There is prior art with Tuttle vs Buttle.

[Terr_]

Ahh, but with Naruto v. Slater, animals can't create or hold copyright, so the art posthumously created by the fly would be in public domain. :p

[jval43]

This is one of the reasons I currently use Gemini for daily use and research.

Google has lots of experience with search history, and presumably handles this better than new companies.

[throwuxiytayq]

This is also the main reason I currently use Gemini. I hate my gmail account, my Android phone is annoying, and I spend too much time on Youtube, so hopefully they take my access away to all of these soon.

[Tenoke]

If you are imagining that, you could imagine it with search doing the same 10 years ago, which would have more thoroughly prevented you from researching things.

[KaiserPro]

what you mean like searching for porn? emailing client lists to your personal email?

[sillysaurusx]

That is in fact what happened to me, except I think the final decision was made by a human since the ban came later. I didn’t issue any queries in between, so I know it was my convo about barbiturates.

[SXX]

It has nothing to do with human decisions. Bans always come later because it's a best way to make sure most people who get banned would never know what are they banned for exactly and what threshold there is,

The same tactics used in game development against cheaters. If it would ban you right after prompt you'll know how to avoid getting banned.

Obviously that didn't worked for you because you wasn't doing multiple attempts to bypass filters like if you were jailbreaking it by repeatedly trying different stuff.

[hobo123]

Like rat poison. If it works too fast, the rats learn to avoid it.

[butlike]

But isn't the whole point to avoid getting banned? Like the system doesn't want to ban you, and you don't want to get banned, so what's the issue with knowing exactly how you get banned?

[charcircuit]

Except in the case of video games it in practice means that cheaters get to terrorize your playerbase without being banned. This tactic is the kind of decision that is made by people staring at metrics all day without considering what's going on in reality.

[DaSHacka]

It's pretty effective actually, this is why they do ban waves.

One of my friends in high school used to cheat on a popular video game. The fact ban waves would occur about once a week to once a month meant whenever his accounts got banned, he never knew why exactly and wasn't able to stop it the next time.

Of course, if ban waves are too long apart then yeah you're just letting a known cheater wreak havoc on the playerbase.

[charcircuit]

But the price of a new account is low. So worst case cheaters can cheat on the playerbase for a couple weeks get banned and then switch to another account and continue cheating for another couple weeks.

[DaSHacka]

That's the problem with many older games, I don't know of a good solution other than either age-gating accounts or keeping the price of the game ~$60