[amelius]

Note that computing the SVD of an m*n matrix is O(mn^2) with m<=n.

So never try to do this for large square matrices. (Just like you should never compute the inverse of a large matrix).

Even for large non-square matrices, computing the SVD can be prohibitively expensive.

[ajtulloch]

Not with randomized methods (which you should almost always use IMO).