I think it's probably got something to do with it. Plenty of non-Catholic denominations and other faiths have had similar abuse scandals, and in the public consciousness the crimes of Catholic priests get filed under "why religion is bad" as much as "why Catholicism is bad".
Well, it's certainly not helping with the recruiting of younger people into that particular religion... Today's 80 year olds in the church probably joined 60-75 years ago.