I'm under the impression the phrase was used because lay Catholics had more days off work since they wouldn't work on Holy Days. Protestants had significantly less Holy Days to the point where some denominations only avoided working on Sundays. Protestants used this phrase as an insult against Catholics for having more days off work. It had nothing to do with freeloading.