Yeah, that makes sense: I brought up stolen password hashes, but I generally disregard keyloggers/MITM/interception because I usually use trusted devices and network encryption (HTTPS), but not all sites do, and I can see how people might be forced to use untrusted devices.
Still, when you've got access to your password manager (to get your password and TOTP token too), you've got access to a trusted device too.
And there is still an option for anyone (including shoulder surfers) to type in your password+token a bit faster than you so they get in: nobody bats an eyelid for getting reprompted for another TOTP token.
You are also vulnerable to someone stealing your password manager password in this manner, especially with a cloud one (which is what most businesses require).
As a conclusion, it does grant you some extra protection against using password only, but when on a separate device, it's really another dimension.