[pjscott]

Not everything parallelizes nicely. Given enough processors, serial computation eventually becomes the bottleneck:

http://en.wikipedia.org/wiki/Amdahl's_law

[aheilbut]

Many machine learning methods parallelize extremely nicely.