I think his point was that if your password is stored on your phone, two factor authentication doesn't actually add any security because it's no longer two factor.
2FA seems a modest improvement at best, especially when it boils down to a TOTP secret you can use anywhere. (I have a greasemonkey script that enters my required '2fa' token for me.) With a yubikey form factor it's much better... It's also relatively useless if you already have a strong password and don't re-use it, i.e. a password manager. Sure it may stop someone from logging in as you if they just have your (unique) password, but if you consider the ways they can just have your (unique) password that doesn't really matter.
So 2FA combines something you have (your phone) with something your phone knows.