[octopod12]

A turing machine is fixed pipeline too. if you define the "pipeline" as its finite set of symbols and states.

and I agree that a folksy sounding title usually hides fuzzy thinking.

but the interesting point made is whether an LLM can figure out if it has "actually" solved a problem or not (?). Technically, this is the classical "halting" problem.

[DannyBee]

I'm not sure the author makes that point usefully, though it is an interesting point to discuss.

I agree this is the classical halting problem, and it can't answer it if the formalism is only as powerful as a turing machine.

Even if you get past that with more powerful formalisms, you will eventually hit the incompleteness theorem with your formalism.

But we can't solve those problems either, so let's put that aside.

You can get very very far in turing machines. Busy beaver programs have demonstrated this:

1. 8000 state (now like 765 or something) turing machine that can't be proven to halt if ZFC is consistent

(this is worse than the halting problem :P)

2. 4888 state turing machine that halts only if there is a counterexample to goldbach's conjecture

3. 5372 state turing machine that halts only if the riemann hypothesis is false

Can an LLM say anything about these? Ever? The first one, maybe not the other two, maybe?

I don't think anything the author goes on about in their post helps figure out if the answer is yes or no.