[panarky]

As millions of people interact with ChatGPT, their writing will subtly, gradually, begin to mimic its style. As future versions of the model are trained on this new text, both human and AI styles will converge until any difference between the two are infinitesimal.

[brookst]

One of the big complaints with LLMs is the confident hallucination of incorrect facts, like software APIs that don’t exist.

But the way I see it, if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should.

I work in PM (giant company, not Python), and one of these days my self-control will fail me and I will open a bug for “product does not support full API as specified by ChatGPT”.

[matthewdgreen]

ChatGPT also thinks certain coding algorithms should violate well-known information-theoretic bounds ;) I’ll put a ticket in with Claude Shannon.

[brookst]

I’m sure he’ll be excited to learn about this gap!

[howlin]

> LLMs is the confident hallucination of incorrect facts

This is a very common feature of delirium in people. Chatting with an LLM seems a lot like what it would be to talk to a clever person with encyclopedic knowledge, who is just waking up from anaesthesia or is sleep talking.

[moffkalast]

Or just the average person on reddit's r/confidentlyincorrect.

[dr_dshiv]

> it, if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should.

Yes! And when it hallucinates references for articles, often times those articles probably should exist…

[visarga]

And if they don't exist, you ask the model to write them from title and link.

[moffkalast]

The year is 2145.

When a new person is born their entire life is hallucinated in its entirety by the all great and powerful GPT. Deviation from His plan is met with swift and severe consequences.

[xiphias2]

Or rather you can just catch method missing in the runtime and patch it with a chatgpt call

[brookst]

Love it. Get ChatGPT to write the missing method, execute it this once, then store it in a file, update the current source file with the include to cache it for next time.

[int_19h]

I can't find it, but someone already did a Python module that plugs into GPT-3 and automatically generates functions on the fly as you call them - and then the same for methods on the returned values etc.

[visarga]

> if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should.

Hahaha, Python language fixing itself!!!

[joe_the_user]

An opposite possibility is that the commonness of ChatGPT will cause people to adopt a style as distinct from it as possible.

Of course, this might mean future Chatbots would successfully emulate that. But it's not impossible an "adversarial style" exists - this wouldn't be impossible to emulate but it might be more likely to cause the emulator to say things the reader can immediately tell are false.

One idea is to "flirt" with all things that people have come up with that AI chokes on. "Back when the golden gate bridge was carried across Egypt..."

[andyjohnson0]

Prediction #1: Once enough ChatGPT output gets posted online, it will inevitably find its way into the training corpus. When that happens, ChatGPT becomes stateful and develops episodic memory.

Prediction #2: As more people discuss ChatGPT online, by late 2023 discussion of Roko's Basilisk exceeds discussion of ChatGPT. (half /s)

[ClumsyPilot]

Or. ChatGPT will overtrain on it's own data and go to shit the way google search did

[visarga]

Training on its own data is a tradition already. For example RLHF example pairs rated by humans are generated by the model. So even our best models trained on their own outputs + rating from human labellers. The internet is a huge rating machine, AI will distill this signal and improve even while ingesting its own text.

[andyjohnson0]

Meta-ChatGPT's loss function optimises for ChatGPT generating training data that maximises the shittyness of Google's LLM.

[LelouBil]

Did you see the new Bing chat ?

#1 is already happening !

See here (other HN thread) : https://twitter.com/tobyordoxford/status/1627414519784910849

[zdragnar]

So long as ChatGPT is forbidden from communicating in certain ways (swearing, speaking ill or positive of controversial people or topics, etc), convergence will never happen. People interact with other people more than they do ChatGPT, so the majority force will remain dominant.

[wankle]

AI will turn half the people into Eloi (https://en.wikipedia.org/wiki/Eloi) while the rest of us will become the Morlocks.

[fullshark]

Sounds accurate and horrifying, I don't get the enthusiasm for this at all beyond a desire to be there first and make a ton of money. All manuscripts get a run through an AI editor, all business writing is even more soullessly devoid of purpose beyond accomplishing task X, all blogposts are finetuned for maximum engagement and therefore ad/referral revenue.

That's already happening I know but it will be amplified to the point that all humanity in writing in lost. All ideas in writing will be a copy of a copy of a copy and merely resemble something once meaningful. Time to go touch grass.

[visarga]

Too much AI for you? You can fix your problem with even more AI! Get your own AI, running on your hardware, loyal only to you. It will act like a firewall between you, the vulnerable, hackable human, and the wild internet full of other AIs. To go out on the internet without an AI is like going for a walk during COVID without a mask, or browsing without AdBlock. Their AI will talk to your AI, and that's how you can be safe.

[onos]

Interesting idea, but isn’t there variance in the output? Eg I’ve seen people ask it to “write in the style of x” etc and different people also clearly have different writing styles.

[Filligree]

You can try, but it isn’t very good at that. The style remains very ChatGPT.

[moralestapia]

+1

We train AIs but they also train us.

[ominous]

Some related idea, in case you like to see that thought explore: https://medium.com/@freddavis/we-shape-our-tools-and-thereaf...

[moralestapia]

Great read, thanks for sharing!