[benreesman]

There’s even a term of art for making a plan up front and then hitting it with a low-skew latent space match: “Chain of Thought”. Yeah, it’s seen numbered lists before.

And if at first you don’t succeed, anneal the temperature and re-roll until you’ve got something that looks authentic.

[AstralStorm]

Right, but is it actually logically sound and complete, rather than truthy?

In short, if you cannot use this thing without validating every step it's worthless for logic. You might as well solve the problem yourself.

[benreesman]

You got me beat: IMHO these things are plenty friggin awesome already and getting cooler all the time. I don't see why there is so much ink (and money) being spilled trying to get them to do things more easily done other ways.

Language models are really good at language tasks: summarization, sentiment analysis, borderline-creepy convincing chatbots, writing pretty good fiction at least in short form, the list goes on and on. At all of the traditional NLP stuff they are just super impressive.

They already represent an HCI revolution with significance something like the iPhone as a lower bound: it's a super big deal.

But while the details are absurdly complicated and the super modern ones represent an engineering achievement up there with anything ever done on a computer, they still fundamentally predict some probability-like metric (typically still via softmax [0]) based on some corpus of tokenized language (typically still via byte-pair [1]).

And when the corpus has a bunch of conversations in it? Great at generating conversations! And when the corpus has some explanations of logical reasoning? Often passably good at looking logical. And when the corpus has short stories, novellas, and novels featuring conversations between humans and science-fiction AIs? Well they can sample from that too.

But imitating William Gibson doesn't make GPT-4 any kind of sentient any more than it makes me a once-in-a-generation science fiction author.

“Real motive problem, with an AI. Not human, see?”

“Well, yeah, obviously.”

“Nope. I mean, it’s not human. And you can’t get a handle on it. Me, I’m not human either, but I respond like one. See?”

“Wait a sec,” Case said. “Are you sentient, or not?”

“Well, it feels like I am, kid, but I’m really just a bunch of ROM. It’s one of them, ah, philosophical questions, I guess...” The ugly laughter sensation rattled down Case’s spine. “But I ain’t likely to write you no poem, if you follow me. Your AI, it just might. But it ain’t no way human.” [2]

[0] https://github.com/facebookresearch/llama/blob/main/llama/ge... [1] https://github.com/facebookresearch/llama/blob/main/llama/to... [2] https://www.litcharts.com/lit/neuromancer/characters/mccoy-d...