You could insist on sticking with the Axiom of Countable Choice if you wanted to avoid some of that.
I'd say it's pretty hard to avoid thinking about 'infinity' though.