[igleria]

I think I've recently read two seemingly contradicting things:

1- LLMs can never generalize theorem proving

2- this paper: "This suggests that contemporary LLMs may already possess rich mathematical knowledge in their parameter space, transforming the challenge from knowledge acquisition to knowledge elicitation"

Not sure what is what anymore!

[bilater]

I think the way to swallow this bitter pill is to acknowledge they can "generalize" because all human knowledge is actually a relatively "small" finite distribution that models are now big enough to pattern match on.

[gmueckl]

Calling human knowledge small is hyperbole. I cannot get any LLM even close to giving accurate answers related to the things I know. They simply do not know what I, a single human being, knows. That's simply because I'm a subject matter expert on somewhat niche topics. There are easily hundreds of thousands of people like me out there.

There's simply no way an LLM can even train on all of that because each bit of true expert knowledge necessarily comically underrepresented in any possible training set.

[whattheheckheck]

Where you you instruct others to go to find out more about those niche topics?

[ashirviskas]

Nice try, AI company AI bot /s

Though I'm not even sure about "/s", it is more than feasible to build such a bot that would gather quality information sources.

[UncleEntity]

Maybe there's a way to reduce the dataset for a LLM to learn to reason down to the smallest possible set and then apply the vast knowledge of humankind on top of that?

I mean, if it can reason about and process the data as it ingests it?

[Davidzheng]

And another way is that the human brain is a relatively "small" circuit that models are now big enough to model ;)

[ak_111]

The LLM can generate the correct search space for the problem, but identifying the solution within the search space is inefficient?

Another way to put this: most of students who study the lecture notes for their high school math already have it within them to get a gold on olympiad (the math itself is not more advance than their high school) but getting a high school kid to get gold on olympiad is hard. It might be something similar to P vs NP.

[wrsh07]

You are going to see a lot of people (both hype and skeptic) tell you things that you can verify. Even while you have a screenshot verifying the opposite of what they are claiming, they will continue to claim it.

For skeptics in particular, you will be able to use a top tier llm and see: does this do the thing someone is claiming it doesn't do? It often will. If you look at recently submitted papers by skeptics you will see them making a claim about state of the art LLMs but then only test using versions from over a year ago (this has happened recently!^)

The way for you to be sure what is what is to just use the thing for yourself and decide what is true.

^ https://x.com/tylercowen/status/1881051976102035880

[solomatov]

You could have a rich mathematical knowledge, while being not very good at proving theorems. Also, you might be good at proving competitive mathematics problems without having a rich mathematical knowledge. It's also possible to have rich mathematical knowledge, and being good at proving theorems but mostly in the field of your expertise.

[sebzim4500]

I think that "LLMs can never X" is just always false.

[solomatov]

LLM can never solve a halting problem (because no one can using a Turing machine).

[woctordho]

A finite-size LLM can solve the finite-size halting problem, and an infinite-size LLM can solve the infinite-size halting problem

[solomatov]

Halting problem input has finite size (i.e. it’s a Turing machine)

[theWreckluse]

"LLMs can never predict the next word"

[euthydemus]

https://x.com/mbalunovic/status/1887962694659060204