[robbedpeter]

And even in the case of billion parameter models, explainability isn't as opaque or mysterious as it's made out to be. You can step through inferences and identify the semantics of individual neurons and clusters and layers. It took a while to see Google deep dream neuron exploration, but you can pick any feature you see in a generated image and identify exactly which neuron/s hold that feature, and how it ended up in an image.

Deep learning and neural networks are giant polynomial equations. Any variable can be investigated and its semantics understood in context of the model.

Gpt-3 neurons can be understood as encoded rules about language. It's tedious and requires a lot of compute and labor. Whether you ever get a map for a model depends on the need for it but there are no technical reasons a "black box" model can't be explained or understood. Neural networks aren't magic, they're just big and convoluted.

Think of it like application disassembly and reverse engineering. On a 100gb densely packed nn model, reverse engineering is going to be difficult, but explainability is just something that can be done if the expense is justified.