|
|
|
|
|
|
by azeirah
1193 days ago
|
|
|
This is pure speculation on my part, but I don't think LLMs in their current form will be able to talk about what they do and don't know. No matter how large the input. There is no introspection in their architecture. Introspection likely has to involve some form of a feedback mechanism and possibly even a "sense of self". These coming years are going to be interesting though. For sure we are going to see experiments built on top of these recent amazing LLMs that _do_ have some form of short-term memory, feedback and introspection! Giving these kinds of AIs a sense of identity is gonna be a strange thing to behold. Who knows what kind of properties will start to emerge |
|
|
So, the information about what it does and doesn't know seems to be there. I can speculate that a limited form of introspection is probably present too: the model needs to know what it will say later to output the current token. A simple example: should it output "a" or "an". To make this decision it might need to model its own state at a later point in time. Of course, I can be wrong.
But I mostly agree with you. Explicit mechanisms for memory and introspection will probably drastically reduce the need for computation power to achieve the same results and they will give rise to more abilities.