[karmakaze]

For me the crux of it is where's the boundary of 'self'. Regardless of how it came to be, we can define a bounded self (whether bio or software) that runs freely taking energy (but not decision input) from outside. Then we can say that that self produces outputs that have internal sources as being 'free'.

To me it basically comes down to infinite butterfly effects where the knowledge of past/current states of the environment can't be effectively used to predict the output of the self.

Another fun way to imagine it is to say 'I' is the complex gut microbiome that's interchanged and evolving and it's the 'source' of decisions more than anything else. It would also solve the problem of how the human population keeps increasing but souls are discrete and countable--even with reincarnation the number of creatures varies greatly. With microbiomes they can be cultivated and/or subdivided, and mixed. Of course this doesn't work out, but it's easy to see there are things that we haven't considered yet to be ruled out.