[vaidhy]

You are correct that I did not define critical reasoning. However, here is an sample case - I set a context and ask the answer from the computer. If the computer rightly understands the context and answers the question, then that is critical reasoning for me. Now that I type it out, it sounds like a version of a Turing test and I do not know if we can do better.

So, in your hypothetical case, I would argue that using the terminology and providing the answers assume some level of intelligence. We get caught up in the training process and automatically assume what computer can infer because it is what we can infer. But the computer is not human and its world model is not the same as our world model. Your fear is reasonable, but religion has gotten away with it for so long :)