"actual reasoning" doesn't mean anything concrete, until you define what you are talking about it can't be the basis of a question you can meaningfully answer.
I think Deutsch in The Beginning of Infinity has a lot to say about this, but I find it really hard to reproduce and summarise. Chapter 7, "artificial creativity" addresses the subject at length, here's Deutsch's own summary of it:
> The field of artificial (general) intelligence has made no progress because there is an unsolved philosophical problem at its heart: we do not understand how creativity works. Once that has been solved, programming it will not be difficult. Even artificial evolution may not have been achieved yet, despite appearances. There the problem is that we do not understand the nature of the universality of the DNA replication system.
He describes personhood, people, as "creative, universal explainers". That's his dividing line between what a child can do, and what LLMs cannot do. We don't know how a child works, but we do know how LLMs work.
While it's true that we don't know how intelligence/creativity/reasoning "works" there's an even more basic problem: It's not a well formed notion.
"The result is that there is no hard problem… You can’t look for the answer to a problem unless you say here are the things I want to answer. If the things you want to answer have no formulation there is no problem.” (Chomsky, about the question of consciousness)."
“There is a great deal of often heated debate about these matters in the literature of the cognitive sciences, artificial intelligence, and philosophy of mind, but it is hard to see that any serious question has been posed. The question of whether a computer is playing chess, or doing long division, or translating Chinese, is like the question of whether robots can murder or airplanes can fly — or people; after all, the “flight” of the Olympic long jump champion is only an order of magnitude short of that of the chicken champion (so I’m told). These are questions of decision, not fact; decision as to whether to adopt a certain metaphoric extension of common usage.
"There is no answer to the question whether airplanes really fly (though perhaps not space shuttles). Fooling people into mistaking a submarine for a whale doesn’t show that submarines really swim; nor does it fail to establish the fact. There is no fact, no meaningful question to be answered, as all agree, in this case. The same is true of computer programs, as Turing took pains to make clear in the 1950 paper that is regularly invoked in these discussions. Here he pointed out that the question whether machines think “may be too meaningless to deserve discussion,” being a question of decision, not fact, though he speculated that in 50 years, usage may have “altered so much that one will be able to speak of machines thinking without expecting to be contradicted” — as in the case of airplanes flying (in English, at least), but not submarines swimming. Such alteration of usage amounts to the replacement of one lexical item by another one with somewhat different properties. There is no empirical question as to whether this is the right or wrong decision. -- Chomsky"
> The field of artificial (general) intelligence has made no progress because there is an unsolved philosophical problem at its heart: we do not understand how creativity works. Once that has been solved, programming it will not be difficult. Even artificial evolution may not have been achieved yet, despite appearances. There the problem is that we do not understand the nature of the universality of the DNA replication system.
He describes personhood, people, as "creative, universal explainers". That's his dividing line between what a child can do, and what LLMs cannot do. We don't know how a child works, but we do know how LLMs work.