he's likely talking bout the internals. sure we know how to train them but nobody knows what the models learn exactly. how those billions of parameters shape the output on inference.
a few months ago, just this year some researchers discovered what might be the neuron that largely decides when to use an in gpt-2. yes 2. that's what he means.
a few months ago, just this year some researchers discovered what might be the neuron that largely decides when to use an in gpt-2. yes 2. that's what he means.
https://clementneo.com/posts/2023/02/11/we-found-an-neuron