No, they're not the same thing! It's not piping to your shell! The shell's single purpose is to execute code. Man is not supposed to do that and it would be considered a huge security issue if it could. In any case, how would you check the downloaded file? With a text editor? Are you sure that can't result in arbitrary code execution?
There is no practical difference. "Nobody" will inspect the man page using a different viewer first. So if I download to disk and then view via man or directly via man is no difference.
A shell script one might inspect first using some viewer. While only few probably do.