Right, but we don't actually view people as block boxes, and neither does 'society', as far as I can see. We care about whether or not someone freely chose to do what they did.
We do? We might pretend we would like to. But we can't even distinguish whether a decision was made with free will or not and we don't even attempt to try and find out.