[hobbyist]

How differently is linear regression than PCA? I understand the procedure and methods are completely different, but isn't linear regression also going to give the same solution on these data sets?

[vicapow]

PCA minimizes the error along the PC axis. Here's a nice article that goes into more detail: http://www.r-bloggers.com/principal-component-analysis-pca-v...

[marcosdumay]

PCA results in a space, while linear regression results in a function within the same space as the original data.

[im3w1l]

To begin with they solve different problems:

Linear regression has a notion of input and output. You want to model how the output depends on the input.

PCA does not. You have a pointcloud and want to find a compressed representation of it.