Companies that are concerned about this can set up SSO authentication with zoom I believe. So once the user is removed from the company’s directory server they wouldn’t have access to the zoom address list either.
I'm not dismissing the overall security point, but this seems like a pretty weak attack vector. If your company is routinely not deactivating accounts associated with your domain as part of your offboarding, being able to see e-mails and pictures of your employees is not your biggest problem.