Even if I am confident that pushing a fat man onto some tracks will stop a trolley and save lives, it is much more likely that I have gone insane than that actually being the case. If this dilemma happened in real life, hopefully I would recognize my insanity and not push the fat man.
This is not an abstract logical problem. If you imagine yourself in the situation, it still feels like taking a risk, and this will influence your answer.