[mrandish]

Agreed. Current frontier LLMs tend to be jagged and brittle in ways which are remarkably hard for humans to reason about. They appear so smart in virtually all the ways humans have traditionally estimated intelligence in other humans (concrete knowledge, verbal fluidity, etc) until you suddenly slam full-speed into a seemingly invisible wall. I just tell people to think of them as 'alien intelligences' which are modeling human behavior from external observation while in reality sharing less in common with us than even mammals or reptiles.