[mNovak]

Unrelated to the accomplishment or proof itself, but it's interesting how much of the prompt, even in this latest-and-greatest model, is spent essentially telling the model to actually solve the problem. Things like "Reject status reports, vague optimism, and claims that an unproved global compatibility statement is 'routine'."

Also a lot prompt spent feeding it strategies, which feel like they should/will eventually be deduced by the model itself, not explicitly stated. That's not to take away from the outcome in any way; rather, it feels sort of like when you would prompt GPT 4, "think through your answer step by step," as a sort of proto-chain of thought.

[futureshock]

I think a lot of this has to do with the post-training these models normally get. They are designed to answer basic questions with straightforward and short summary answers. They have the capacity to reason deeply, but they are not biased towards that unless prompted. I think it's because LLMs as they are in 2026 are both highly capable but also parlor tricks. They are not sentient, you just set them up with the context and then they roll downhill. You could reach a genuinely novel answer, but only with the right input. They have no will and depend on human guidance. They are both a marvel and a machine.

[walrus01]

Something I've noticed is that if you run Qwen 3.6 35B-A3B (Q8) with a low temperature of 0.4, and leave default reasoning turned on, it will spend quite a lot of time in reasoning/thinking mode. But often it does figure out how to solve something on its own by correcting itself within its reasoning loop before it outputs the final 'answer'.

If you watch the progress of the reasoning in llama-server while it's doing the thinking, you can track its progress. Sometimes the dead ends it goes down or things that it considers and then disregards are themselves something useful to re-prompt it with later, and send it 'rolling downhill', to use the metaphor of another commenter here, in another direction towards the same effort.

Putting 3.6 35B-A3B into a state that lets it spend a lot of time in its reasoning mode before outputting an answer is probably not something that a web based SaaS LLM would tolerate, because it would frustrate many of the non technical end users who want a LLM to spit out an answer now.

[DiabloD3]

You'd have less problems with 27B, btw.

[walrus01]

I didn't mean so much that it was a problem, but actually in some projects for exploring what's possible, watching its "thinking" mode output at temp 0.4 is intentional and useful to take notes and begin exploring new directions. Sometimes it'll come up with something I hadn't thought of, perhaps it will go down that direction, perhaps it'll disregard it...

[gb2d_hn]

'roll down hill' is a good way of putting it. They don't have 'will', but that's as we want it I think. I think alignment is harder if they develop will. Without will they are still tools that feel like an exoskeleton rather than something that will control us.

[singularity2001]

Agents have a state which will unfold as a plan, especially in planning mode. Why not call this 'will'?

[gb2d_hn]

I think because without the initial prompt, they are only interpreting our will, and do not act under their own volition.

[DiscourseFan]

Even Fable hallucinates. I had it tracking down some very obscure Ancient Greek inscriptions and the response just made up a translation/context for one inscription after "looking it up." Now, it was still a very particular thing and I really had to get into the weeds to push it to that point, but who knows how many other gaps, near or far, it will happily skip over just for the sake of coherence. I think this is an issue more primarily with LLMs than sensory systems like Waymos or all the ML applied to industrial processes--that really only requires pattern recognition, often very impressive and subtle pattern recognition but its no different from an artist learning to tell the difference between Prussian blue and Navy blue or a Sommelier learning the fine distinctions between various regions of Bordeaux. Language has many more avenues and introduces inherent contradictions that do not always lend themselves to easy resolution. But there are no alternatives paths visible to the models, there is only ever the next word; stochastic, in the sense that the possibility space is open; deterministic, in the sense that the final response is always a necessary result of every token that came before it in their total sequence. Thus, any response is constantly in the work of erasing any possible alternative, slowly narrowing down what can be written. If contradictions in language necessarily involve interpretation, then the models will only ever choose one at a time, and for them, it will always be the right one. But anyone who understands the subtleties of language can tell you that when it comes to determining the truth of an indeterminate statement, there is never just one right answer; or, rather, the answer which is taken to be the "right" one depends on the possibility of its own reversal into falsehood, if any argument has to be made to justify it.

[HarHarVeryFunny]

Hallucination is fundamental to how LLMs work, and is mostly unrelated to how large or smart they are.

Everything that an LLM outputs is just a statistical language-based (no real grounding) prediction. Luckily with a model based on a large training set most common questions may elicit coherent responses from the training data, but you don't need to veer too far off into "questions less asked" territory to get responses based on training data mashups that amount to best guesses that are wrong, aka hallucinations. The unfortunate part of this is that as a user you may only catch this when asking a question about something you are already fairly knowledgeable about, then you give some pushback to the model and it cheerfully acknowledges "you're right - I made that up".

[trogdor]

>Even Fable hallucinates

It’s in the name :)

[HarHarVeryFunny]

Yes, the prompt, and use of subagents is interesting. It could be characterized as tree of thoughts rather than "think step by step" chain of thoughts.

I see the need for this as coming down to two things:

1) LLMs are fundamentally prediction machines, and therefore ultimately will only do what they are prompted to do (and whatever that leads to). They may have been trained on, and/or have access to, all sorts of information that may be useful to solve a problem, but their predictive nature is to only use that information if explicitly prompted to, else it remains "dark" and inaccessible other than by luck. You're essentially having to tell the model "solve this problem using techniques A, B & C", otherwise techniques A, B & C will be off the radar unless the model already associates them to the problem.

2) The fundamental reason this sort of brute force tree-of-thoughts "explore all avenues" prompting is necessary, is because the model itself has no inherent curiosity to explore. Humans work differently. Our behavior is also prediction based, but we are also built for problem solving and continual exploration/learning via traits like curiosity (driven by prediction failure).

Problem solving via search can to some extent be prompted for, as here, or achieved via an external harness, but impasse resolution via curiosity, directed exploration and continual learning (if/when something new/unpredicted is encountered) is trickier. You can't usefully prompt a predictive model to "be curious" since that will only cause it to predict what a curious person would do, rather than the model being curious in reaction to the specific gaps in it's own knowledge.

[famouswaffles]

It might also be helpful more than 'necessary'. The 2 most notable solutions have come from open ai themselves, but most of the 'LLM solves open problem' category are from 3rd parties doing their own thing.

This one was pretty impressive in its own right (probably the most impressive outside these 2), and the prompt is concise and basic.

https://www.scientificamerican.com/article/amateur-armed-wit...

https://chatgpt.com/share/69dd1c83-b164-8385-bf2e-8533e9baba...

[kypro]

I thought that too. The prompt is full of metaheuristics.

I remember a couple of years back when people were saying how prompt engineering was a skill, and reading this prompt kinda took me back to that.

Were I to guess, the reason the model couldn't do this itself is because most of the time, for most problems, a lot of this is bad advice.

In search optimisation you're often trading between time and quality. A very broad search will return very bad results for a long time. Where as a more depth oriented search with some heuristic will tend to return a pretty good result (if not optimal or close to optimal) quickly.

I'd assume models naturally want to find some middle ground there because that's the best thing to do most of the time, but for very difficult problems where a decent attempt isn't good enough you want a much broader search that doesn't have the time constraints. Much of the prompt seemed to be in that direction – really encouraging broadness of the search, preventing early convergence, and remove pressure of time constraints.

[sudo_cowsay]

Same. I remember something like using AI to optimize your prompt to that specific model helps a lot. I am currently trying it and can sort of see a difference (I think....).

[rando1234]

It's funny, I found exactly the same thing when I asked about P=NP. The models outright refused to attempt to solve it, claiming it was too hard. I had to really battle to get it to suggest some promising suggestions.

[scarmig]

LLMs have basic reasoning and a whole lot of memorization. Through that basic reasoning and pruned search, combined with piles of compute, you can prove lots of things. But the memorization of human failure prunes that possibility, and you need to expend effort convincing the LLM not to prematurely prune based on previous human failure.

[riddlemethat]

The current foundational models have basic reasoning with glimpses of brilliant reasoning.

[krona]

LLMs have almost no fluid intelligence or capacity for abstract inductive reasoning, relying on crystallized intelligence for basically everything.

[singularity2001]

If anything their capability of abstract inductive reasoning is way beyond the average human given how much better LLMs are at solving math problems, it's the paradox that they can do complicated reasoning before they can do intuitive peep-pe-boo.

[krona]

abstract inductive reasoning requires dynamic learning in unfamiliar contexts. All the benchmarks which measure this specifically (in particular, task composition) see LLMs fail catastrophically. There is quite a lot of research published on these limitations now.

[imdsm]

I think they have some fluid intelligence, if somewhat brittle

[ankit219]

llms are good at greedy depth first search[1]. so prompt / human needs to tell the model the breadth parts to take.

and human writing the prompt here did a lot more than that. asking to include parallel-edge 2-cycles, admitting disconnected graphs, specifying the emptyset cover, multiset counting, it all likely came from previous fake proofs.

[1]: see: ankitmaloo.com/fable - its the way most llms are trained, and is also natural owing to autoregressive nature.

[kzrdude]

I guess "depth first" is just an expression of so called chain of thought, which is just a linear sequence. I don't know if GPT has any search-like (tree search) algorithms in their reasoning, it would be quite interesting if they did (they probably have researched this area, at least).

[krona]

LLMs don't have the executive control to dynamically manage and compose mental sub-routines, when it has never seen those sub-routines in its training corpus. That requires fluid intelligence. ARAOC benchmark measures this specifically.

[stingraycharles]

And all this is while they themselves proclaim that GPT 5.6 is much better at understanding intent and doesn’t need this type of guidance. Yet they themselves still do it, which I consider a more powerful signal.

[cbm-vic-20]

At some point, we will no longer be mathematicians and engineers, but instead become matrix psychologists. "Do not make mistakes."

[ouraf]

Optimism and status reports burn extra tokens and make the user more prone to ask the model to process the problem again because it was "so close to solving".

This way you get more profit per API user and subscription users reach their quota faster and are contacted to update their plan to a higher tier.

It's working exactly as intended

[giancarlostoro]

Sounds to me like motivation is what LLMs need to do the impossible, I guess Kamina was right all along:

“Kick logic out and do the impossible! Remember that, that’s the way Team Gurren rolls!”

The sweet irony is all the jailbreak style fixes could hamper this approach.

[geraneum]

Maybe in previous failed attempts that what the model landed on and they’re preemptively stopping it. Did they release the any info on the failed attempts?

[linzhangrun]

Maybe models also need a specially tuned version for mathematical research, just like "gpt-5.3-codex".

Looking forward to "gpt-5.6-mathx".

[nullbio]

This is the key to training better and better models.

You collect all of the direction prompts and anti-direction prompts, and you finetune the model to behave that way for the particular domain, under those particular conditions, as a default.

This is also why I don't believe for a second that the AI labs aren't training on our data, even when we opt out. This sort of data collection is a requirement for improving the models.

It's also why Anthropic purposefully gives you a dumb model or a corner cutting model to goad you into guiding it into the correct answers, so that you do the fine-tuning for them. OpenAI at least, does not do that. Which is why ChatGPT is far less infuriating to work with.