[md_]

One point not mentioned by others here:

In the case where recovery entails a required password change and the site imposes some password history--the common case, though I don't know if this is true of Fastmail--the real account holder is guaranteed to notice if a recovery event happens, whereas ordinary password+SMS-interception may not be noticed.

So, yes, for the most part this is as weak as SMS second-factor--but it's less likely to go undiscovered by the real user.