[rosstaylor90]

I disagree, 10 years ago AIs nailing these types of competition would have been seen as very impressive. The fact goal posts can move on this now shows how much AI has progressed.

(Also the term “approximate retrieval” is a bad one - reasoning is inherently a process of chaining together associations. What matters is whether the reasoning reaches the right conclusions. Still some way to go, but already very impressive in tasks traditionally considered harbours of human reasoning!)

[CamperBob2]

I disagree, 10 years ago AIs nailing these types of competition would have been seen as very impressive.

It would have been seen as witchcraft.

[bossyTeacher]

> What matters is whether the reasoning reaches the right conclusions

no, it doesn't. a broken clock is right twice a day, reasoning is about the journey more than the destination

[rosstaylor90]

RL has more than two steps...

[bossyTeacher]

Point is that reasoning is more about the conclusions. if your steps are wrong, your reasoning is wrong regardless of the conclusion. Poor reasoning is what could make an LLM conclude that 1 + 2 = 3 but what 2 + 1 = [some number other than 3]