I decline to accept ads because I don't want professional manipulators tricking me into acting against my best interests. I consider myself fairly good at resisting manipulation, but these are highly skilled experts backed by the full power of modern neuroscience, and they aim to catch me off guard. Adverts are hazardous to view, and you should not do so without utmost mental focus and discipline, which is impractical during general browsing.
The avoidance of annoyance is a just bonus.