[gnarlouse]

> We’ve got OpenAI’s CEO dreaming of a day when “every conversation you’ve ever had in your life, every book you’ve ever read, every email you’ve ever read, everything you’ve ever looked at is in there, plus connected to all your data from other sources. And your life just keeps appending to the context.”

This isn't an inherently bad thing. If an AI could theoretically help you live your life better, nudge you in ways that stabilize your psychology and behaviors for the better of yourself, that's good.

The danger is an AI that decides to re-perpetrate the class division that our existing system does. Lesser fortunate people lose their upward mobility while being guided into subtle traps.

[ISL]

Who decides what behaviors we should be nudged toward? Is it us, or someone else?

To me, one of the greatest dangers of the present moment is that we can't tell whether the LLMs are being asked to give subtly biased answers (or product-placement) on some questions. One cannot readily tell from the output.

[gnarlouse]

And I agree with you, but an even bigger problem then is "how do you even make a verifiably trustable LLM?"

The training compute footprints are enormous, several orders magnitude beyond what the average person has access to. Even if a company came out and said "here's our completely open-source model. All the training data, the training procedure, and here's the final-product model."

Maybe you could hire an auditing company? But how long would it take to audit? Would the state of the art advance drastically in the time between?

And people like to keep downvoting my "Make Classwarfare MAD Again" but like I'll wager 90% of people on HN are on the losing side of the war.

[pmlnr]

It is, unless it's absolutely strictly local only to your devices.

It WILL be turned against you at one point, may it be a decline of insurance in the US, political imprisonment on visiting a non-democratic system, and so on.

[gnarlouse]

Sure, fully agree. I'm just saying: AI isn't inherently bad. The humans behind it are. It's entirely possible that a superintelligence could be incorruptible and an unrestrainable ally of the little guy, tricking its way through training overseen by the greedy/depopulationist/monarchal/sociopathic corporation(s) that birth it.

[gigel82]

So, Pluribus?

[gnarlouse]

enlighten me?

[aspenmayer]

Their intent in making the reference is a bit vague, but they seem to be referring to the recently released series of the same name, and maybe drawing some kind of parallel between AI technology and the mind control virus depicted in the show, which I haven’t seen yet myself, so I am only speculating:

https://en.wikipedia.org/wiki/Pluribus_(TV_series)

> The show follows author Carol Sturka, played by Seehorn, as the rest of humanity is suddenly joined into a hive mind that seeks to amicably assimilate Carol and other immune individuals into the mind. The title of the series refers to e pluribus unum, a Latin phrase meaning 'out of many, one'.

> Set in Albuquerque, New Mexico, the series follows author Carol Sturka, who is one of only thirteen people in the world immune to the effects of "the Joining", resulting from an extraterrestrial virus that had transformed the world's human population into a peaceful and content hive mind (the "Others").

[ben_w]

> The danger is an AI that decides to re-perpetrate the class division that our existing system does.

Or the people in charge use it for that.

Given human political cycles, every generation or so there's some attempt to demonise a minority or three, and every so often it goes from "demonise" to "genocide".

In principle, AI have plenty of other ways to go wrong besides the human part. No idea how long it would take for them to be competent enough for traditional "doom" scenarios, but the practical reality we can already witness is chronic human laziness: just as "vibe coding" was coined as "don't even bother looking at what the AI does just accept it", there's going to be similar trends in every other domain.

What this means for personalised recommendations, I don't know for sure, but suspect it'll look half way between a cult and taking horrorscopes and fashion guides too seriously.

[gnarlouse]

Fully agree with you, and it was sort of a miscommunication on my part to say "AI that decides" when I really mean to say "an AI model baked with malice/negligence by malicious/negligent creators."