| > I wasn't intending to raise concerns over emergent consciousness or similar Oh jeez, then we may have just been talking past each other. I thought that’s what you were arguing for. > That just isn't important here at all. It is, though. The fact that the underlying processes are well understood means that, if we so wished, we could work backwards and understand what the model is doing. I recall some papers on this, but can’t seem to find them right now. One suggested that groups of weights relate to specific kinds of high level info (like people) which I thought was neat. > the comparison wasn't intended they wouldn't be called "artificial intelligence" Remember “smart” appliances? Were we meant to compare an internet connected washing machine to smart people? Names are all made up. I do actually think AI is a horrible name as it invites these kinds of comparisons and obfuscates more useful questions. Machine Learning is a better name, imo, but I’m not a fan of personifying machines in science. Too many people get sci-fi brain. |
I definitely agree on the term machine learning - it seems a much better fit but still doesn't feel quite right. Naming things is hard, but AI seems particularly egregious here.
> The fact that the underlying processes are well understood means that, if we so wished, we could work backwards and understand what the model is doing.
I'm not sure we can take that leap. We understand pretty well how a neuron functions but we understand very little about how the brain works or how it relates to what we experience. We understand how light is initially recognized in the eye with cones and rods, but we don't really know exactly how it goes from there to what we experience as vision.
In complex systems its often easy to understand the function of a small, more fundamental but of the system. Its much harder to understand the full system, and if you do you should be able to predict it. For LLMs, that would mean they could predict a model's output for a given input (even if that prediction has to account to randomness added into the inference algorithm).