I don't remember like 98% or 99% of my passwords. I have something like 270 on my private accounts and probably 300 passwords on my work accounts. Well password manager is useful and I can always use pw reset option built in systems.
I kindly propose everyone to forget all their passwords.
Then they mostly don't need second factor if they generate random password each time and don't care about remembering them at all.
How is that? Everybody living in my house can get my Yubikey yet doesn't know my password. If I get robbed, my bank account is still (relatively) safe.
There are a lot more people far away from you than there are close to you. If breaking your security requires physical proximity (such as to steal a yubikey), then you are much safer just based on this. It's also easier for people to blindly steal credentials for millions of people online than it is for them to steal millions of physical security keys.
Alternatively, passwords are commonly reused across websites, so a failure of any of those websites can lead to a compromise of all of them, which is not the case with a YubiKey. Along that same line of thought, passwords are phishable, where YubiKeys are not.
It's also possible that people in your physical proximity could shoulder surf your password, install a keylogger (which could be a physical keylogger, if you normally use a USB keyboard, not just software), or use a strategically positioned camera to do some digital shoulder surfing. Passwords aren't immune to trust issues when it comes to physical proximity. Ideally, you trust those you are near to some extent.
YubiKey also has a fingerprint-protected device coming out soon[0]... which would raise the bar for the threat model in this discussion some. Using a fingerprint and/or PIN to unlock a YubiKey preserves most of the benefits, while eliminating most of the concerns that people are mentioning. HSMs can choose to self-erase after a certain number of failed PIN attempts, so even a short PIN is not something that can easily be brute forced without an unpatched vulnerability.
If websites would allow you to only use any one of your YubiKeys to authenticate (obviously meaning you can have multiple, with backup YubiKeys stored somewhere safe in case you lose your main one), I think that would be a significant improvement in security over password authentication for most people. This is basically what the WebAuthn standard is attempting to do. I don't expect most people to be interested in buying 3 security keys and carrying one around all the time, though.
For the last bit: If it's suitably seamless, it's actually not that bad. I've been carrying one on my keyring, and it's just another key, only this one "unlocks" websites.
Most people in your home are not trying to hack you.
A lot of people outside your home are trying to hack you.
Shifting your exposure from "everyone in the world with an internet connection" to "people who are in/near your home" greatly reduces your risk, objectively.
Most people won't purchase and use a Yubikey either though. Really just depends on your threat model, if remote attacks or local attacks are of higher risk. An obvious improvement would be the use of both a password and physical security token.