Noob here (with his own mail server so I'm not familiar with these kinds of issues). What's the problem with a bounce? You pay for the delivery attempt anyway I assume, why do they care?
Mostly to do with making sure their service isn't being used for spam, to protect reputation. Bad reputation can mean blacklisting which is a bad time when your service is centralized on emails.
Sure, but how would allowing bounces lead to being used for spam? If it bounces, that means the mail was not delivered. Why are these errors bad for a reputation? It doesn't even matter whether the mail was spam or not, since it wasn't delivered anyway. That's what I don't get, of course I know they're trying to keep their reputation good.