[nyrikki]

Turing and Gödel proving that whenever a mathematical system is rich enough to describe the arithmetic we learn at school, it cannot prove its own consistency is a bit of a barrier.

ML is sophisticated pattern matching and finding.

As we know that even defining simple rules of arithmetic is impossible, how will pattern finding do so.

Sure some company could pay slave wages to make LLMs better at it, but correctness is important in math and LLMs have no concept of truthfulness.

[Dylan16807]

> As we know that even defining simple rules of arithmetic is impossible

That's not what it means. You can evaluate systems, you just have to do it from outside.

> how will pattern finding do so.

What are you talking about? We're not asking the computer to invent its own number system.

[nyrikki]

Feed forward Neural Networks, like LLMs using attention are effectively Directed acyclic graphs, so it has little it can do to 'evaluate from the outside'

Even if you can move to second-order logic logically-valid formulas in second-order logic is not recursively enumerable. AS RE problems are those which a Turing machine can answer "YES" in a finite amount of time, but a "NO" answer might never come, moving out of RE spaces like DAGs is not computationally feasible.

The problem with 'pattern finding' is that it can learn some even HoL problems if it is in the corpus, but will fail on even simpler problems that weren't.

Here is one paper that explains this. https://arxiv.org/abs/2301.09723

"A persistent problem in corpus-based ML, in all its applications, is that the patterns that the AI finds do not actually reflect the fundamental characteristic of the problem, but rather superficial regularities in the training data, known as “artifacts”.

LLMs aren't doing the 'logic' of the math problems, they are finding patterns in their training data that are hopefully close enough to work for the presented problem.

This is why you can use AI to say learn about intervals on the real line, as those are of finite VC dimensionality, but algebra questions that are outside of it's corpus tend to be very difficult for LLMs to be correct on.

And obviously issues like the Entscheidungsproblem don't magically go away because we have a tool like ML that is far more computationally efficient than brute force, but still insufficient.

As LLM's will confidently present wrong answers as correct, how is that helpful for students?

[Dylan16807]

> Feed forward Neural Networks, like LLMs using attention are effectively Directed acyclic graphs, so it has little it can do to 'evaluate from the outside'

I thought you were talking about the math that's being taught.

No, you can't prove the tutor is mathematically consistent. Is that supposed to be a problem? No tutor in the history of the world has ever met that standard.

No matter what you think of the current state of LLMs, that bar is so high it's meaningless.