[esafak]

They are not interpolating, which is what I think you meant to say, except for a loose definition humans would meet too.

What do you think of the latest thinking models, and what is your test of thinking?

[Salgat]

An LLM is one very big nonlinear regression used to pick a token with a clearly defined input, output, and the corresponding weights. It's still far too straight-forward and non-dynamic (the weights aren't constantly changing even during a single inference) compared to the human brain.

As far as the latest "thinking" techniques, it's all about providing the correct input to get the desired output. If you look at the training data (the internet), the hardest and most ambiguous problems don't have a simple question input and answer response, they instead have a lot of back-and-forth before arriving at the answer, so you need to simulate that same back-and-forth to arrive at the desired answer. Unfortunately model architecture is still too simple to implicitly do this within the model itself, at least reliably.

[esafak]

Learning and thinking are separate things. Today's models think without learning -- they are frozen in time -- but this is a temporary state borne of the cost of training. I actually like it like this because we don't yet have impenetrable guardrails on these things.

> If you look at the training data (the internet), the hardest and most ambiguous problems don't have a simple question input and answer response, they instead have a lot of back-and-forth before arriving at the answer, so you need to simulate that same back-and-forth to arrive at the desired answer. Unfortunately model architecture is still too simple to implicitly do this within the model itself, at least reliably.

Today's thinking models iterate (with function calls and Internet queries) and even backtrack. They are not as reliable as humans but are demonstrating the hallmarks of thinking, I'd say.