The authenticator apps are just as weak as SMS against phishing, which is a gazillion times more common than sim swaps.
The authenticator apps cause loss of credentials more often because people don't back them up and then drop their phones in the toilet. The thing that makes SMS weak to sim swaps is also the thing that keeps a mountain of people from losing access to their accounts.