[qmalzp]

I think the first point is only true for symmetric matrices (which includes those that show up in multivariable calc). In general, the eigenvectors need not be orthogonal.

[soVeryTired]

Yep, you could well be right. The image of an ellipse under a linear transform is definitely an ellipse, but I'm not sure about the eigenvectors in the general case.

The symmetric case is by far the most relevant for probability theory though.

[tprice7]

In general it's the eigenvectors of the positive-semidefinite (hence symmetric) part of the left polar decomposition.