[tomp]

I don't want this to develop into a flame war, so this will probably be my last reply in this thread...

I studied math. I love math proofs. It's just not what I would expect if I would sign up for a ML class. Note, however, that I'm not complaining about the contents of the ML class, which I think were awesome, a really nice broad overview of the ML techniques. I really think I learned a lot. I'm just complaining about the homeworks, nothing else. They were not ML homeworks, they were CS homeworks (particularly, Matlab class homeworks). I don't know the contents of the real Stanford class, but believing your description, the same kind complaint applies.

You're wrong when you say that "lazy solutions" don't perform well in the real world. In the real world, robots aren't programmed in Matlab (I hope). In compiled languages, loops are just as fast as vectorized code (which is implemented using loops). Vectorized code having better performance than imperative looping code is just an artifact of the fact that Matlab (and e.g. R) are interpreted languages. (Sure, there might be some additional optimizations possible on vectorized code, like SIMD instructions, cache locality, special algorithms for sparse vectors/matrices, but this is in most situations premature optimization. Being able to write correct code and read it later beats 2% gain in performance!)

[nessus42]

The reason for all the math proofs in the real Stanford class is, I assume, because it is training you to have a deep understanding of Machine Learning, and perhaps even to become a Machine Learning researcher. This is the type of education that I received at MIT, and there's much to value in it. If you can do all these proofs, then you probably really understand things. On the other hand, it can be brutally difficult, and I still have stressmares when I sleep to this day over this brutally difficult style of learning.

If you really want to understand ML, you should do the real class. All the material is online, so you could do it at your own pace.

The online class, however, was a watered-down version of this, and designed to give you a taste of the real class without being brutally difficult. Perhaps it will inspire some people to do the work of the full-blown class.

On the other hand, sometimes you just want to know how to apply some technology to real-world problems without having to understand all the gory details. I think the online AI class is more along these lines. They had a final project, for instance, that was structured as a contest to see whose solution worked the best. I think they used canned ML packages instead of coding them on their own, but then wrote their own code to apply the canned ML software to some sort of open problem. Perhaps you would enjoy that class more.

As to getting only a 2% performance gain from vectorizing your code, I'm skeptical of this claim. I know that highly respected people (e.g., Guy Steele) have claimed that people still use Fortran, despite its drawbacks, in part because of it superior performance over C in being able to vectorize code and getting substantial performance gains. Furthermore, a vectorized algorithm is more easily adaptable to a Hadoop cluster, or a GPU, and both of these can lead to huge performance gains.

In higher-level languages, such as Java, vectorized solutions can also lead to much higher performance, as you can use a native linear algebra library, which will, no doubt, perform better than Java loops.

Furthermore, if you can turn an algorithm into linear algebra, you will have a much more maintainable solution, as you can turn a program that is several pages long into one that is several lines long. That is surely going to be easier to maintain!