My point is that there’s no scientific way to demonstrate that one has made a choice vs that outcome being determined (perhaps probabilistically) or being random. We can’t “go back” and do it a different way to demonstrate we’ve actually chosen.
We don’t understand what it means to be intelligent in the way people generally mean, and we don’t understand (scientifically) what it means to choose. So we can’t use choice to usefully define intelligence.
>My point is that there’s no scientific way to demonstrate that one has made a choice
"I'm sorry, your honor, there is no scientific way to demonstrate that I have made a choice to shoot the victim dead. Therefore, I am not guilty!"
The way that an AI gets a data set to be trained on is via a choice made by humans. You can't reason your way out of the fact that a choice is made, that it's not made by the AI, and that it has consequences.
We don’t understand what it means to be intelligent in the way people generally mean, and we don’t understand (scientifically) what it means to choose. So we can’t use choice to usefully define intelligence.