[turzmo]

Both impressive and terrifying. But as always, the methodology is buried: how many open problems were tried until they found a success?

If they tried this on 1000 problems and this is the one that succeeded, it still means that there are 999 open problems that an LLM cannot one-shot. It seems likely that this would remain the situation until the next model.

If this is the first one they tried, maybe we’re totally hosed.

The conclusions are so different in these cases that it is impossible to know what to think. Though it is reasonable, I think, to assume that a company is willing to push the maximally misleading narrative —- especially a company known for questionable ethical direction at the top, and one that is still circling an IPO, and one that is in the tech industry, where conjuring an illusion of growth and progress is sufficient for success.

[emil-lp]

> But as always, the methodology is buried: how many open problems were tried until they found a success?

Not only that, but they have like 500 world-leading experts in mathematics and IMO alumni, so how do we know one of the agents wasn't hardcoded to return a proof that the mathematicians had found?

I'm a mathematician/graph theorist, and I've tried ChatGPT 5.3, 5.4, 5.5, and now 5.6 on a bunch of simple-ish open problems, and I've never gotten a solution.

[turzmo]

I’ve had a similar experience in physics. Excellent domain knowledge and semantic search, but the intellectual sparkle and reasoning just isn’t there and it still often throws out a lot of wrong ideas. It is very useful for coding.

But there is a discrepancy between (the implication behind) these reports and how I subjectively feel talking to LLMs. Granted, I don’t have access to whatever cutting edge model is out there for as many credits, but I also don’t feel like I’m talking to an IMO silver medallist.

[i000]

What is a "simple-ish open problem". I guess it falls under "solving this open problem in mathemathics is left as an exercise to the reader" ;)

[taegee]

They have to feed the bubble.