[distant_hat]

People don't understand what exponential improvement means. GPT-3 is a 175B with B parameter model. Another few rounds of doubling and we could be seeing models spit out short stories and novellas.

[derivativethrow]

Have you read the GPT-3 paper? The authors are pretty forthright about how the GPT techniques likely won't scale much beyond 175B parameters.

[ur-whale]

Yet, the darn thing still can't reason.

[visarga]

They "can't reason" but they can solve symbolic integration and differential equations (https://arxiv.org/pdf/2006.06462.pdf), beat us at all board games, and more recently get to be almost impossible to tell apart from human writing (GPT3).

[jbay808]

Five years ago we would have said "the darn thing still can't write a cohesive paragraph".

[wnoise]

That's still true. It can write a paragraph that's usually grammatical, and can stay on topic, but it's missing things like facts, or even the ability to remember which side of an argument it's taken previously.

[jbay808]

I think over the course of several paragraphs that's true, but within one it tends to be pretty good.

[distant_hat]

People look at the present and feel like that's all that's there. You don't look at a 2-year old and say he barely makes any intelligible sounds yet.

[Enginerrrd]

To be fair, it wasn't trained to.

To be really fair, I think you need to more precisely define what you mean by "reason". It absolutely CAN reason by some measures.