[firebaze]

I'm not sure if it is feasible to provide all relevant sources to someone who doesn't follow a field. It is quite common knowledge that LLMs in their current form have no ability to recurse directly over a prompt, which inherently limits their reasoning ability.

[dr_dshiv]

I am not looking for all sources. And I do follow the field. I just don’t know the sources that would back the claim they are making. Nor do I understand why limits on recursion means there is no reasoning and only memorization.

[light_hue_1]

This is just totally false.

That's exactly what countless techniques related to chain of thought do.

[llm_trw]

The closest explanation to how chain of through works is suppressing the probability of a termination token.

People have found that even letting llms generate gibberish tokens produces better final outputs. Which isn't a surprise when you realise that the only way a llm can do computation is by outputting tokens.

[dr_dshiv]

It’s sometimes like, are these critics using the tools? It’s a strange schism at the moment.

[llm_trw]

It's my job to build these tools. I'm well aware of their strengths and shortcomings.

[dr_dshiv]

Unless you are building one of the frontier models, I’m not sure that your experience gives you insight on those models. Perhaps it just creates needless assumptions.

[llm_trw]

I'm building the layer on top of the models.

People call it agentic AI but that's a word without a definition.

Needless to say better LLMs help with my work, the same way that a stronger horse makes plowing easier.

The difference is that unlike horse breeders, e.g. Anthropic and openAI, I want to get to the internal combustion engine and tractors.