[math_dandy]

We have no definition of reasoning that is sufficiently precise to be useful.

But we do have a bunch of benchmark tasks/datasets that test what we intuitively understand to be aspects of reasoning.

For AI models, "being able to reason" means "performing well on these benchmarks tasks/datasets".

Over time, we'll add more benchmarking tasks and datasets that ostensibly test aspects of "reasoning", and people will develop models that succeed on more and more of these simultaneously.

And these models will become more and more useful. And people will still argue over whether they are truly "reasoning".