[Vespasian]

It is but it also shows current limitation of the reasoning skills of this models.

A super human AGI would easily be able to use tools like Wolfram Alpha and also derive most of it on the fly or from memory just by thinking about it.

If you set your expectations to AGI (right now) you will be disappointed but that doesn't mean it's not immensely useful.

Unfounded hype and real technological progress seem to go hand in hand.

[flakeoil]

Should Einstein not be allowed to use a calculator and instead waste his time on performing the calculations by hand?

[Vespasian]

Obviously yes. And how they use these tools is important.

But it's quite interesting to see what these models can do internally and what they can't do yet. It possibly outlines future research areas (beyond " more scale") and opportunities for competitors to enter the market (which is usually better for consumers and society as a whole).

Maybe that's the difference between productization and academic reasearch.