[glifchits]

Part of the introduction really reminds me of Nassim Taleb's Antifragile.

"As artificial agents enter the human world, the demand that we be able to understand them is growing louder."

"Let us stop and ask: what does it actually mean to “understand” another agent? As humans, we face this challenge every day, as we engage with other humans whose latent characteristics, latent states, and computational processes are almost entirely inaccessible. Yet we function with remarkable adeptness."

In the chapter I'm referring to, Taleb describes situations where practitioners outperform academic theories. Practitioners are driven by results, they develop heuristics and mental tricks with experience, but they don't always have a good understanding of the underlying complex system that they're predicting about, and they can't usually explain their heuristics and tricks.

Academia (and broader society) tends to demand an explanation for why things behave a certain way. Usually this means developing some nice-sounding narrative that may or may not be true. I think it's fair to say Taleb is skeptical that it is at all possible to learn the underlying mechanisms of a complex system to the point where we can actually predict the future behaviour of that system.