[Crash0v3rid3]

I've given up on these LLMs.

The amount of fatigue I get having to determine if what they tell me are fact is just too much.

I'm sure someone will tell me my experience should be similar with generic web search, but at least I'm in control of what websites to read through to determine sources.

However, I'll agree with most that state it is helpful for creative purposes, or perhaps with coding.

[fferen]

I've found they serve almost exactly the opposite purpose as search engines. When I want reliable info and don't need hand-holding: search. When I have no idea what to search, or want a quick intro to something: ChatGPT. Together, they are very powerful complementary tools.

[simonw]

Your experience absolutely shouldn't be similar to generic web search. The idea that they are an effective replacement for that is one of the most widespread misunderstandings.

They're good at SO MUCH OTHER STUFF. The challenge is figuring out what that other stuff is.

(I have a few examples here: https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-... )

[unsui]

> The challenge is figuring out what that other stuff is.

Unfortunately, the major problem is something you pointed out in your blog post:

> We must resist the temptation to anthropomorphize them.

The reality is that, we in meatspace simply cannot help but anthropomorphize them.

These language models regularly pass the Turing Test (admittedly for low bars).

They are surprisingly good at bypassing the Uncanny Valley to hit the sweet spot of persuading without legitimate justification, simply because they are so convincing in formulating sentences in a manner that a confident human would.

Yes, these tools have legitimate use cases (as you outlined in your blog).

But the vast majority of use cases will be those of confidante, of discourse partner, of golem brought to life without understanding what exactly has been brought to life.

That's really dangerous.

[isthiseasymode]

That explanation makes me think of blockchain.

I do think AI is already more useful that block chain has ever been, however.

[roflc0ptic]

Blockchain is good for one narrow thing most people don’t care about: reconciliation in multiparty transactions.

LLMs appear to have myriad uses, today, no Twitter .eth con men required.

[shagie]

I find it very useful for doing zero shot and few shot classifications of natural language input.

The "use it as a chat companion" is an interesting technology demo that demonstrates some emergent processes that make me wish I was back in college on the philosophy / linguistics / computer science intersection (though I suspect the hype would make grad school there rather unpleasant).

[quantiq]

> They're good at SO MUCH OTHER STUFF. The challenge is figuring out what that other stuff is.

I’m getting Déjà vu

[simonw]

The difference this time is that we've figured out all kinds of stuff that this is useful for.

The challenge genuinely is helping people learn how to use it, not finding those applications in the first place.

[Xelynega]

I don't think we've figured out stuff it's useful for, we've just created tech-demos that are much more digestible.

For blockchain/crypto companies their tech demos have required you having a wallet, downloading an app to interact with the chain, or just having lackluster visuals for the users involved in the tech-demo.

On the other hand, LLMs can be interfaced via strings in APIs, so it's braindead to spin up a text-interface for those APIs and no wallet setup or learning about new chains, the English that works on one model will work on another and produce results that are better than most cryptocurrency/blockchain tech-demos.

Notice that none of this relies on us having "figured out all kinds of stuff that this is useful for". We've made cool looking tech demos that make it easy for anyone to generate content.

Much like blockchains I feel it's the underlying technology that's actually useful(distributed PKI for blockchains and deep learning networks for GPT), and GPT itself is only 'useful' insofar as it's an easy-to-interface with implementation of a much more powerful idea.

[simonw]

I'm talking about directly useful things you can do just using ChatGPT, without even writing code against the API. I have a few examples here: https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-...

[travisjungroth]

> I don't think we've figured out stuff it's useful for, we've just created tech-demos that are much more digestible.

This is out of date. Many people are using ChatGPT frequently for real things. It’s totally different from blockchain.

[saurik]

I mean, the usual argument about why blockchains aren't useful implies they have to be useful for every person in all situations and that tradeoffs are unacceptable, so if there is some marginal extra cost or complexity then no matter how many benefits I might claim to be getting from using blockchain technology every single day as a replacement for random banking institutions I'd previously been having to deal with for decades that I'm somehow just wrong and there are no actual use cases...

..and that's the same deal for GPT as far as I can tell: you might think you are getting value out of it, but people such as maybe-literally-me are going to whine that the error rate is high and that people are not paying enough attention to how they are using it and that at the end of the day it is probably worse for you than learning how to do things yourself and that the whole thing is overrated because many of the things people try to use it for can be done by a person and maybe we should regulate it or even ban it because all of this misuse and misunderstanding of it are dangerous to the status quo and might be the downfall of western civilization as we know it.

To be clear: I'm using it (ChatGPT) occasionally for some stuff, but it hasn't replaced Google for me anymore than crypto has fully replaced banks... and yet the fact that I am using either technology as often as I am on a daily basis would probably have been surprising to someone 10-15 years ago. And yet, in practice, most of the stuff people are excited about in both fields is, in fact, a tech demo more than a truly useful product concept, and one that only is exciting momentarily until you get bored.

[Sivart13]

With Google search going steadily downhill, I find it really tough to verify anything that ChatGPT authoritatively states is true

Everyone on here is so enthusiastic about AI gobbling up the entire software landscape, I would just like a search engine that has any chance of telling me if something is factual

[alfalfasprout]

It's going to be even worse when search results and training data from these LLMs is just output from other LLMs.

[klyrs]

Product idea: the original PageRank over the Wayback Machine dataset pre-2022, with a mechanism to establish trust in users to moderate and cull SEO.

[eastbound]

SciHub ;) Not joking, any search engine will pervert the results, like Google, like OpenAI said they would, “to protect the children”.

[ChatGTP]

Buy an encyclopaedia and put it on your desk because it doesn’t sound like it’s going to get better anytime soon.

[kolbe]

I've had your same experience. I've found them mostly to be an error-prone search engine, with somehow less accountability than the open internet, because it hides its sources.

At least with Stack Exchange answers, we have who wrote it, what responses there were, what the upvoting behavior around it was. And for the most part, I've found ChatGPT will transcribe often times wrong answers very poorly.

One small example, I asked it to solve the heat equation (i useded the mathematical definition, and not "the heat equation") with dirac initial conditions on an infinite domain. It did a good job of recognizing which stack exchange answer to plagiarize, but did so incorrectly, and after a mostly correct derivation, declared the answer was "zero everywhere."

[simonw]

Somewhat surprisingly, language models are TERRIBLE at mathematical or logic puzzles.

[svachalek]

It's kind of interesting that our science fiction projected traditional computing's strengths, math and logic, into the AI future with overly logical and mechanical AI characters. But our first creation of fully communicative AI has elementary school strength in these areas while it's probably better than the average adult at writing poetry or an inspiring speech.

[kolbe]

That's a whole other topic.

I was mostly commenting on how it just plagiarized a correct answer off of Stack Exchange, except it took an incorrect hard right turn at the end to make up a solution.

[simonw]

What makes you think it was copying information from Stack Exchange in this case?

[kolbe]

This was me just testing it. I was aware of the particular SE answer ahead of time, and it followed the whole thing close enough that I had assumed it had internally mapped to it. But I suppose it didn't have to be that way.

[broast]

It's like dealing with electricity (or maybe the internet). Early skeptics believe it is a curiosity with little application. People see how it can jump all over the place and create disasters that they can't imagine having engineered systems to finely control its behavior and create reliable complex functions and become the bedrock for computing.

[pstorm]

I think there is also an aspect of willful disregard. This technology may change a lot, and it may be easier to dismiss that idea rather than process it.

[ChatGTP]

Do you think there might be the opposite going on ? Wanting to believe something that isn’t there because you won’t have to do as much work, feel smarter etc ? Because it’s really hard not to anthropomorphize it ?

Gloss over all the incredible dangers we might be exposing our world too just because it’s “fun to play with” and see what AutoGPT can do to the Internet ?

[typon]

I use it for thing's that don't really matter if they're exactly correct. For example, coming up with a travel itinerary for a country I have never visited. Rewriting a work email with better English. Summarizing a news article. There are lot of things that don't require ultimate precision. I feel like people expect these models to do something they aren't really designed for - and the mismatch in expectations causes people to be let down. They are just tools - not "mildly conscious beings" like OpenAI founders wants you to believe.

[baq]

They aren’t search engines or knowledge databases. They’re language computers. Use them for computing on language.

[pmarreck]

Are you speaking of ChatGPT 3.5 or 4?