One reason for this is that ads generally have a high dynamic compression applied to the audio to make them seem louder. It’s jarring because the content you’re watching has a more reasonable level of dynamic compression.
That's a good point, but I think I'd still find the audio annoying even at low volume. The images... now, those are sometimes interesting to watch, muted.