[mirker]

Agreed.

Here’s the thing: the authors of that paper got early access to GPT-4 and ran a bunch of tests on it. The important bit is that MSR does not see into OpenAI’s sausage making.

Now imagine if you were a peasant from 1000 AD who was given a car or TV to examine. Could you really be confident you understood how it worked by just running experiments on it as a black box? If you give a non-programmer the linux kernel, will he/she think it’s magical?

Things look like magic especially when you can’t look under the hood. The story of the Mechanical Turk is one example of that.

[ux-app]

>Could you really be confident you understood how it worked by just running experiments on it as a black box

the human brain is a black box, we can certainly learn a lot about it by prodding and poking it.

>Things look like magic especially when you can’t look under the hood.

imagine we had a 100% complete understanding of the mechanical/chemical/electrical functioning of the human brain. Would knowing the magic make it any less magical? in some sense, yes (the mystique would be gone, bye bye dualism), but in a practical sense, not really. It's still an astonishingly useful piece of grey matter.