[thelazydogsback]

I'm tired about hearing about the lack of "explainability in AI" when the issue is with deep-learning & non-symbolic AI, the very allure of which to date has been its black-box, just-train-it nature, leading to all kinds of neat-o scary (truly) high-dimensional text and image trickery without the foresight of actually bothering to try to actually understand anything. We've had some pretty incredible explainable AI, planning & NLP processing, etc., in the past (1970's to early 2000's for the most part) that needs to act as the "prime motivator" to drive the DNN methods that really excel at processing at the periphery.

[rolisz]

And what results did those methods from the 70s to early 2000s get? Why are they not used?

Deep learning has obtained good results in a couple of fields (NLP and CV) and nothing else comes close to it.

[robbedpeter]

And even in the case of billion parameter models, explainability isn't as opaque or mysterious as it's made out to be. You can step through inferences and identify the semantics of individual neurons and clusters and layers. It took a while to see Google deep dream neuron exploration, but you can pick any feature you see in a generated image and identify exactly which neuron/s hold that feature, and how it ended up in an image.

Deep learning and neural networks are giant polynomial equations. Any variable can be investigated and its semantics understood in context of the model.

Gpt-3 neurons can be understood as encoded rules about language. It's tedious and requires a lot of compute and labor. Whether you ever get a map for a model depends on the need for it but there are no technical reasons a "black box" model can't be explained or understood. Neural networks aren't magic, they're just big and convoluted.

Think of it like application disassembly and reverse engineering. On a 100gb densely packed nn model, reverse engineering is going to be difficult, but explainability is just something that can be done if the expense is justified.