[rglover]

Worthwhile read. It helps to learn (or confirm) how things are working under the hood. What's interesting is that understanding how it all works makes it clear that all the hype around models "thinking" or being "sentient" is just marketing fluff for "the math works and it's really impressive how that translates to human-like cognition."

[CamperBob2]

The interesting thing is how it becomes harder and harder over time to draw a line between what human brains do and what these models do. Once you have seen chain-of-thought reasoning in action (does the book even cover that?) you realize that defense of human superiority is strictly a "God of the gaps" game. Arguments that work today won't hold up tomorrow, because they're built on unreliable definitions, special pleading fallacies, obviously-temporary tech limitations, and general hand-waving.

It's not that the technology is magical or special, it's that we're not. That being said, finding new ways to study the nature and limits of cognition and consciousness after 2000+ years of unproductive navel-gazing feels very magical and special.