Could you explain why? Is it because a log-in request essentially looks exactly the same as a "forgot password" request and is likely to slip under the radar of someone monitoring for suspicious activity?
If you use N sites which all adopt this authentication mechanism (i.e. widespread adoption); and if I can access your e-mail, then I can access all of those N sites. Furthermore, yes, because all those accesses look normal, nobody would detect it as unauthorized.