If the recruiter doesn't ask me first (or I don't agree to a meeting), this is called "spam", and I would be happy for the system to just not allow it.
I have never encountered a situation where recruiter starts immediately with an invite without prior conversation (such invite also blocks the time slot of the sender - it would be stupidly ineffective to do that). It is hypothetical and improbable scenario that is not even worth mentioning here.
It just doesn’t make sense to do it ahead of time in such situations. Email client could simply ask if I trust the email before processing the attachment (and some clients do that). Automated pre-processing of attachments is a general risk that doesn’t apply only to calendar.