As I said in my first comment, a recovery numbers turns it from 2FA into single factor. Obviously that is less secure, since a phone number is generally less secure than a password. I'm talking about true 2FA over SMS.
Recovery numbers should be stored somewhere safe, like your house. If your attackers have access to your house they could beat you up with a baseball bat instead of going through the trouble with finding your recovery codes and phishing your passwords.
Recovery numbers are recovery telephone numbers. If they sim-swap you, they have your telephone number. The end. Virtually all email accounts and online services tie themselves to your damn telephone number, and there are tens of thousands of people in each phone companies who can move your number.