Hacker News new | ask | show | jobs
The biggest risk of large language models is that they may bury us in lies (scientificamerican.com)
44 points by garymarcus 1278 days ago
10 comments

I think this will rapidly accelerate a set of closely related trends that were rapidly growing in the internet: curation, trust, and reputation.

In a world with the massive aggregating ability of google, amazon, Facebook, etc, we've become deprived already of truth. Whether its the alto-junk filling up search results, fake reviews on cheap knockoffs on amazon, or the latest mis-information campaign on Facebook, it becomes hard to sort through the endless options (curation), trust the reviews, and find reliable sources.

People already search on reddit for some review, because they don't trust Amazon reviews, but what happens when reddit becomes GPT playgrounds? You'll see a further rise in organizations that either sell Curation (Target, Costco), reputation (Apple App Store) or sell Trust (Consumer Reports, Wirecutter). People flock to influencer YouTubers for product reviews, because its harder to fake, and people will build new business around such solutions.

> I think this will rapidly accelerate a set of closely related trends that were rapidly growing in the internet: curation, trust, and reputation.

But will it scale?

Who knows but that might not be the most important factor.

> One of the most common types of advice we give at Y Combinator is to do things that don't scale

I’m not going to explain someone else’s quote, but in my perspective, in a world of limitless abundance, perhaps trying to do everything isn’t the winning strategy.

If Amazon has every product ever made available (and 75% of it is cheap drop shipped junk), there’s extra value in being the trustworthy brand that doesn’t have everything, but instead the right things.

If Twitter is full of bots and misinformation, then there’s value in being a small mastadon instance of hand curated friends where you can trust everyone.

If Google can index every website, and most of it ends up as GPT lies, then being a librarian that can vet sounded for you is immensely valuable.

http://paulgraham.com/ds.html

Also something to note is that the focus of the article is preventative as opposed to creating new outlets. Just because reuters is there doesn't mean everyone will read it, but a good number of people could see content that's not as reputable

> If Twitter is full of bots and misinformation, then there’s value in being a small mastadon instance of hand curated friends where you can trust everyone.

I agree with this, but the way that we use the internet and media might not be the way that most people use it. If the focus is preventative, I have no idea how curation could scale in a way that prevents bad actors from reaching an audience, just because there's so much stuff to curate

Something to keep in mind with this article (and the myriad of other articles like it): the author has a psych background, not an engineering background. He's worked as a consultant, not an engineer. He has a lot of things to say about AI

He calls for changes in AI, but never does he talk about concrete implementation (in this article or elsewhere). People like the author of this article, let's call them "thought leaders", what are they actually good for? What do they build? It seems like all they can do is make noise and spend money

> Systems like ChatGPT are enormously entertaining and even mind-bogglingly human-sounding, but they are also unreliable and could create an avalanche of misinformation.

It sounds like it could be used as a “flood the zone” machine in some ways, and led me to wonder whether this capacity was a feature or a bug.

Lies are just a special case of an uncountable infinity of impossibilities. It will only make humans who lie have an existential crisis. People who believe in a consistent world can carry on normally.
3500 years ago, the Assyrians and Babylonians made it policy to always lie (propagandize) about their victories. They had no official losses, they only won closer and closer to home...

There is nothing new under the sun.

A cheap way to replace politicians.
How models like GPT-3 could accelerate the production of misinformation, and what we might do about it
> they may bury us in [the wrong people's] lies

Fixed the headline.

Truth itself is an imaginary construct. All animals are evolved to survive not to seek truth. “Truth seekers” are performing in a status game with the objective to prove genetic strength to prospective mates (however unsuccessful that quest might be). Even if we could know the truth, it would be a waste of valuable resources that could be better spent elsewhere.
How do you know what a non-wasteful allocation of those valuable resources is without truth?

An accurate understanding of reality is certainly not the only factor in achieving fitness & status, and I'd even believe at a certain point increased resolution when it comes to understanding reality has diminishing returns (and am aware of some research that supports this).

But absent understanding of reality, there is no effective sustainable fitness or status (or effective allocation of resources).

I don't know what you'd call that understanding but truth works for me.

You have to trust your own body and mind to be healthy enough to keep you going - and if it can’t trust others to support you. None of that requires Truth. But definitely to avoid jail time you better understand what Truth is in your place of residence. And that is to realize it’s bullshit - if you buy into the lies you’re going to be a mark.
> You have to trust

You may want to look into the etymology of "trust." That's one of several indicators that you're actually relying on the concept of truth in your comments here -- there's really no getting around that if you're going to make assertions about how things are. Even if they border on addled word salad.

Want to argue that assertions of truth can be tied up with power? OK. I get that and have argued it myself. Want to argue that grasping the truth is complicated and can be very difficult? Sure. Want to argue for an epistemology where "truth" is nothing more than an imaginary construct? So are 0 and i and yet at the same time they're clearly reflections of graspable aspects of reality.

Want to argue that truth is nothing more than a power grab? This can itself be a move related to power, and probably is. Furthermore, it's just not sustainable. To make an assertion is to make a statement of truth.

>How do you know what a non-wasteful allocation of those valuable resources is without truth?

In a post-truth world, any allocation of resources is acceptable.

True statements matter. If it didn't than Boolean statements wouldn't work. Duh!

Also, is research not ultimately meant to be truth seeking? Should we not explore to understand world, for example the laws of physics? Or is that truth not useful and a waste of valuable resources. In that case - you should probably get off the internet since it apparently isn't very valuable to you.

True as in a Boolean statement is a way of organizing experience rather than direct access to some hidden, metaphysical object. Your brain has to interpret stimuli to tell whether a Boolean statement is true or false - or interpret stimuli from a computer. You can learn something about the necessary conditions of experience by carefully analyzing the process and separating what is subjective from what is objective, but that isn’t accessing “Truth”.

There’s a giant gap between what this article is talking about, which is Truth, than the true statements from discrete mathematics. Discrete math is rooted in necessary conditions of experience; this article’s Truth is rooted in power.

Life is a competition, so if a rival tribe leveraged truth to build a better weapon, your tribe, that's "not wasting resources" on truth, would go extinct, just like those animals.
You’re on to the issue and almost figured it out. Why are the tribes fighting in the first place? They have to be told that the other side is evil - either because they are attacking first or responding to it. Who is responsible for communicating that Truth and why are they doing it? The issue of weapons as a tool for cultivating traumatic experience does require this Truth. But the physics of the weapon are based in objective experience which is something else entirely.
"The other side is evil" is not a truth, it's a lie to deceive the dumb masses who swim in the sea of emotions and barely engage their minds. Truth doesnt change and is the same for everyone.
"Truth is that which, when you stop believing in it, doesn't go away"
I don’t believe in CNN or FOX but they seem to keep on chugging along, so I definitely agree on one level. But if everyone stopped believing in them, their advertising revenue would drop enough to put them into bankruptcy.
And we know the truth of this how ?
It’s not Truth because it’s not from a position of authority - I am just a random commenter. You’re asking the wrong question. The right question would be, “Would agreeing with this make my life better or worse?”
We’ve been in a post-truth society for some time now. Anything that makes it easier to lie is a positive in my book. Society needs to accept that we’re well past the time where information can be trusted. Either we adapt or die.

One thing that helps me is to understand that I really don’t need to know the truth. I wake up. I see my family. I watch my daughter smile. I enjoy the company of friends. I drink a nice glass of wine. What value does the “truth” have in my daily life? Most of the time, what is true and what is lie only comes up in a political discussion, and it almost always is unpleasant.

> I wake up. I see my family. I watch my daughter smile. I enjoy the company of friends. I drink a nice glass of wine. What value does the “truth” have in my daily life?

When people talk about privilege, this kind of bucolic normalness is what they mean; all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run. You can go to the store and buy wine without ever once having to test it for antifreeze.

> You can go to the store and buy wine without ever once having to test it for antifreeze.

This doesn't say anything about truth, only past experience. Which isn't truth

> all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run.

It depends on trust, yes. But trust doesn't depend on truth (because someone doesn't need metaphysical or religious beliefs for them to be trustworthy)

> all of those things are great, but their ability to stay great is conditional on a huge, precise system that depends on trust (which in turn depends on truth) to run.

I 100% agree that the system is built on trust and that the system is where I derive my enjoyment of "bucolic normalness."

However, trust in the system has absolutely nothing to do with truth. All systems are driven by those with power. The opinion of the masses, whether informed by the truth or otherwise, doesn't matter.

For instance, another person mentioned the lie of "de-nazifying Ukraine." Russia has also plainly expressed its imperialist ambition and the ability to reduce NATO's influence in Eastern Europe. Both of those things are true. Does that mean Russia is justified in invading Ukraine because they told the truth about their intentions? The population in Ukraine is strongly against Russian influence. But if those in power wanted Russia's influence, the population would be powerless to stop the transition. To see evidence of this, look no further than Belarus. In Belarus' 2020 election, something like 90% of the population voted against the president. 5% - 10% of the population actively protested in the streets. Workers went on strike. And yet, the elites in Belarus aren't interested in regime change so nothing happened.

Take gerrymandering as another example. The goal of gerrymandering is very plainly to disenfranchise certain voters. Everything knows that. It is the truth. And yet gerrymandering still happens.

Another example: do you really think Trump, the senators, and representatives responsible for the Jan. 6th insurrection really believed that the election was stolen? Of course not. They wanted power and "stop the steal" was just a pretext. I'm sure many of the "stop the steal" supporters understand this too. The lie is just a pretext to get what they want. Again, the truth doesn't matter. You're not going to convince Jan 6 supporters of "the truth" and have them change their mind, because they're motivated by a selfish grab for power. They could care less about the truth.

Ultimately, trust in the system comes from the shared belief that our representative form of government is the best for everyone. Sometimes, that means needing to sacrifice a personal objective, like a quest for power, for the good of the entire system. If the elites want a different form of government, like one where they consolidate power for themselves, then that is what will happen. That is what's happening. The truth isn't going to stop it.

> You're not going to convince Jan 6 supporters of "the truth" and have them change their mind, because they're motivated by a selfish grab for power. They could care less about the truth.

Someone is more at fault if you prove that a wrong was committed because of malicious intent instead of outright ignorance. People who want to see punishment have an incentive to do this. This is why people try to disregard the insanity defense in court when the defendant is obviously insane. The defendant is insane and "in need of support" until they're sitting in front of a judge, then they're suddenly not

But of course, this doesn't contradict the general point you're making

True. I guess my point is that those who fear for a loss of truth are generally mistaking malicious pretext for ignorance.
One only has to look at the Russian invasion based on “de-nazifying” Ukraine or global warming denialism or big tobacco spending decades obfuscating cancer or or or four thousand other post truth things to see that good people ignoring and being okay with lies leads to bad things.
You’re fighting a pointless war.
So, what? Give up, grin as the world burns?
No, we learn how to adapt to a world where truth is not relevant.
It’s better than Surrendering to lies
That’s the thing. You’re not surrendering to lies. You’re just choosing not to play.
If someone tells you the wine you had was poisoned, is it not important if that is a lie or the truth? I imagine it would.
Would it? If you tell me you poisoned my wine I wouldn’t give it a single thought. Or is there more to this analogy that you’ve neglected to mention?