[taco_emoji]

It is literally just a goddamn language model. it is very good at making plausibly human-like sentences. It is not a general intelligence, it is not your friend, it is not a research assistant. It is not designed to deliver content which is correct, it is designed to deliver content which is similar to human language.

It might get things correct most of the time! But that is purely incidental.

[kevin_thibedeau]

It does subsume a corpus of factual information. I use it as a search tool for topics and relationships that traditional search engines can't handle. You just have to know that whatever it outputs isn't trustworthy and needs to be verified.

[jstarfish]

Part of the corpus is explicit bullshit though, and we don't know to what degree. It internalized conspiracy theory and WebMD alike. In a generative capacity, it only reliably produces fiction. Ever. Fictional stories often take place in realistic settings and reference real facts. They sound real. But they're still fictional compositions.

Using GPT as a reference to anything is the same as using a Michael Crichton novel as a biology reference. It looks right, but why would you waste your time asking questions of something you can't trust and have to double-check everything it says anyway? Nobody would keep an employee like that around, nor would you hang out with someone like that. It's friendly enough, but it's a pathological liar.

There's too much black magic going on inside the black box. We don't know how prompts get tampered with after submission, but it might be worth it to pepper "nonfiction" tokens in prompts to ensure it skews on the right side of things. It certainly responds to "fiction" when you're explicit about that.

[dekhn]

yes but it's still very much just a language model, not a knowledge model.

[travisjungroth]

It is literally just a goddamn electric motor. it is very good at converting chemical energy into mechanical energy. It is not a universal engine, it is not your horse, it is not your servant. It is not designed to create movement which is correct, it is designed to create movement which is similar to a piston engine.

It might move you forward most of the time! But that is purely incidental.

[verdagon]

I believe electric motors _are_ actually intended and advertised to create movement which is correct, and come with warranties and liability about reliability. (Or maybe I missed a joke in here somewhere?)

[seydor]

now imagine believing that this electric motor can self-drive itself to the supermarket

[travisjungroth]

ChatGPT isn’t just an LLM. It’s literally not. There’s a web server, interfaces, plugins, etc.

LLMs are this super powerful thing (like a motor) and people are getting to play around with it before it’s fully harnessed. There’s this strange phenomenon where because it’s not totally harnessed, people just rip on it. I don’t know if they think it makes them sound smart, but it sure doesn’t to me. It’s like seeing a motor on an engine stand and being like “But the crankshaft rotates. I want to go in a straight line! This isn’t a transportation solution and anyone who thinks so is just naive. And horsepower? Stop zoomorphizing it!”

Sure, there are lots of problems. But don’t mix up the limitations of this component of the system, the current limitations of the system overall, and the possible capabilities of the system. Someone builds a car that stalls when it drives through a river and people point out that an ICE engine needs air and how could you even think about driving through a river? Then someone else is off making a snorkel.

[vba616]

I think you've got hold of the wrong end of your analogy.

If you just hold it the right way you will be illuminated and we will all agree how relevant it is.

If someone announces to the public, they have a revolutionary new motor that is going to transform personal transportation and solve climate change...

...but it is presented with a chat interface - the motor itself is directed through typing text and it demonstrates its motion/power through text messages in response - rather being connected to a vehicle or generator, rather than a transmission, then it's BS beyond a reasonable doubt.

Even though there is no logical or technical reason preventing such a setup, it's just not how professionals do things. If the motor were so damn useful, then people who like to use motors would use it for the things that they do use motors for!

There is no logical or legal requirement in science to use LaTeX Computer Modern fonts...but as a layperson if I can't tell if someone is a crank or doing advanced physics, I'm going to be exponentially more skeptical if they are using Times or Palatino.

[travisjungroth]

Judging them for having a chat interface is the exact type of thing that I think is unintelligent. It would be like… judging an extremely popular paper for their choice of font. We’re past that.

[vba616]

Who is past it? Is AI past it? "It" = processing the world in terms of correlations?