[pmoriarty]

"we do understand that they generate words with probabilities one by one though"

This is no more helpful in understanding AI's than is knowing that human brains operate according to the laws of physics is helpful in understanding the human mind.

[skybrian]

Yes, there are important mysteries that need to be solved through research [1] but there are still some limits we can deduce from how they are built [2].

[1] https://skybrian.substack.com/p/dont-settle-for-a-superficia...

[2] https://skybrian.substack.com/p/ai-chats-are-turn-based-game...