[decimalenough]

> it can work through the complex, vague machinery of reason with more scaffolding

No, it can hold more floating point numbers.

I'm not an expert in the field, but I've yet to see a solid rebuttal to this paper;

https://arxiv.org/abs/2401.11817

[atleastoptimal]

A claim that LLM's can in a theoretical sense be 100% accurate all the time is not the same as the claim that scaling models with more compute/params will reduce hallucination. The former is a far stronger claim and I agree with the paper in that it probably isn't the case, but we don't rely on general reasoners (a.k.a. humans) to be 100% accurate all the time either.

> No, it can hold more floating point numbers.

Fallacy of composition. Just because an LLM is made up of floating point numbers doesn't mean its capabilities are limited to that of bare floating point numbers, in the same way that the individual faculties of a neuron don't preclude the human brain from emergent properties born from the synthesis of its synapses.

[decimalenough]

You're the one who started with the "complex, vague machinery of reason with more scaffolding" here. I'm simply pointing out that that's not actually a thing: it's just floating point numbers.

[evrydayhustling]

That paper shows hallucinations can't be eliminated, due to approximation error. But it is completely compatible with hallucination becoming less probable as scale reduces that approximation error.