[nroman]

When I was studying this in college I always found the "Eigenfaces" example very enlightening (http://en.wikipedia.org/wiki/Eigenface).

In case you're not familiar with them, the basic idea is treating a image of a face as a very high dimensional vector, and then doing what amounts to PCA on a collection of them. I'm leaving off a few steps, but the resulting eigenvectors converted back into images helped me grasp what was going on in a much more intuitive fashion.

[Homunculiheaded]

About a year ago I repeated this experiment (well a very simplified version) with MegaMan sprites, including a quick write up of the process for anyone interested: http://willkurt.github.io/EigenMan/

[neumann]

This is how we introduce it to our undergrads. Using the faces is a great way to demonstrate model reduction and snapshot method as well as a glimpse into one of the algorithms used in face recognition software.