The reason why they are indistinguishable stems from our ignorance to learn about a deeper abstraction. We cannot simulate further than what we already know.
Not entirely. I am simply stating that, our simulations are indistinguishable because, 'we' are simulating them. The simulations aren't going to get any better than what 'we' understand.
That being said, it doesn't imply we have a perfect understanding of the Amoeba down to the last particle of it. (since we absolutely have no idea what it is).
Yes, agree with your point on how they become indistinguishable.
And, w.r.t RNNs, yes, probably we don't understand how they learn, but has that aided to process of how visual perception actually happens?
We can tell when our simulations behaviour doesn't match the goal. But when it does, then what?
You seem to imply there is some (actual, literal) magic we are missing.