[kuboble]

I have asked chatgpt reasoning model to solve chess endgames where white had king and a queen vs king and a rook on a 7x8 chessboards. So to compute value for all positions and find the position which is the longest win for white.

Not creative, not novel and not difficult algorithmic task. But it requires some reasoning, planning and precision.

It failed miserably.

[AlexCoventry]

o4-mini-high did make an error I had to point out, on the first attempt: https://chatgpt.com/share/680eeea2-264c-800e-8497-3903ea6309...

[throwaway4aday]

I think you need to be more specific about which "chatgpt reasoning model" you used. Even the free version of chatgpt has reasoning/thinking now but there are also o1-mini, o1, o1-pro, o3-mini, o3, and o4-mini and they all have very different capabilities.