Hacker News new | ask | show | jobs
by LoganDark 45 days ago
I would say you communicate to the model and you interpret the model's outputs. I would not say the model communicates back though.

I'm not sure that models are complex enough to have a consistent internal representation of a concept the same way that organic brains can to communicate. I'm not sure of any quantitative science backing this up though. Models don't know anything across iterations yet.
1 comments
blackoil 45 days ago
> Models don't know anything across iterations yet.

Can you expand. They have both context and memory?

link
LoganDark 45 days ago
Most models do not have any persistent state or output that is separate from their input. They consume a stream of tokens and then output a probability distribution. The probability distribution will always be the exact same for that particular stream of tokens. There is no internal state, thoughts, mood etc., only prediction based on the input. "Memory" is usually just something injected into context by the harness and updated by usually a tool call from the model.

I'm sure there are research prototypes that work differently from this but I haven't seen any enter the mainstream yet.

Also, diffusion language models have a different evaluation order but I think they also do not really have internal thoughts or feelings because they also do not seem to have any sort of hidden state that encodes anything like that.

link