[BiteCode_dev]

Just yesterday people downvoted me on HN for suggesting ChatGPT had the best ability to deduct things, because, according to down voters, it didn't have any.

But the "dumb parrot" narrative that was quite fitting on GPT2, and maybe still useful for understanding the foundation of the tech in GPT3, is now just a demonstration of ignorance.

We don't know what they are really doing. Nobody working outside of openai knows.

But if you use it long enough, it will be obvious we have passed the fancy completion phase, there is a little logic in there. Not a lot, but it makes all the results much better already.

And definitely much better than the competition.

[exe34]

> Nobody working outside of openai knows.

I wouldn't be too sure openai people know either. if a machine has enough moving parts, it's hard for an individual human to understand all of it. nowadays it's possible that nobody quite knows why the silicon compiler put a particular block in a particular position on the die. it just figured that's the best way to save power or space or whatever.

[w1nk]

For a more informed opinion than folks on the internet, here's some work from Microsoft with early/internal access to gpt4: https://arxiv.org/abs/2303.12712 . I don't think people close by these systems share the same dumb parrot sentiment at all.

[giobox]

I've never been sure what to make of that paper. It was published by Microsoft shortly after Microsoft's big deal with OpenAI and reads a lot like a marketing piece to me. Many of the observations didn't reproduce the same way once ChatGPT4 in public hands too. If nothing else, I'd prefer it come from a party who hadn't just signed a $13 billion dollar deal with OpenAI a few weeks prior with a view to using their products to sell more new products/features... It's somewhat self-serving for Microsoft to argue ChatGPT4 is super-awesome/sparks of AGI etc, regardless of validity of claims.