[cubefox]

If it solves a problem, we generally don't know how it did it. We can't just look at its billions of weights and read what they did. They are incomprehensible to us. This is very different from GOFAI, which is just a piece of software whose code can be read and understood.

[kreetx]

Any statistical model does this.

[cubefox]

Statistical models have just a few parameters, machine learning models have billions. Possibly more than a trillion.

[kreetx]

The number can be anything, is there a number at which "we don't know" starts?

The model's parameters are in your RAM, you insert the prompt, it runs through the model and gives you a result. I'm sure if you spend a bit of time, you could add some software scaffolding around the process to show you each step of the way. How is this different from a statistical model where you "do know"?

[cubefox]

For just a few parameters, you can understand the model, because you can hold it in your mind. But for machine learning models that's not possible, as they are far more complex.

[kreetx]

So a 150 parameter model we "don't understand how it works"?