| Here's the problem: There is no silver bullet in Machine Learning and many of these approaches (SVMs, Neural Nets, Random Forests, PGMs, etc.) have their pros and cons that depend on many variables, for example: - How much data do you have wrt dimensionality? - How "easy" do you suspect your problem to be? Is it likely linearly separable? Equivalently, how good are your features? - Do you have many mixed data? Missing data? Categorical/Binary data mixed in? (Better use Forest, perhaps!) - Do you need training to be very fast? - Do you need testing to be very fast on new out of sample data? - Do you need a space-efficient implementation? - Would you prefer a fixed-size (parametric) model? - Do you want to train the algorithm online as the data "streams" in? - Do you want confidences or probabilities about your final predictions? - How interpretable do you want your final model to be? etc. etc. etc.
Therefore, it doesn't make any sense to talk about one method being better than another. One thing I will say is that, as far as I am aware, Neural Nets have a fair amount of success in academia (which should be taken with a grain of salt!), but I haven't seen them win too many Kaggle competitions, or other similar real-world problems. SVMs or Random Forests have largely become the weapon of choice here. Neural Nets do happen to be very good when you have a LOT of data in relatively low-dimensional spaces. Many tasks, such as word recognition in audio or aspects of vision fall into this category and Google/Microsoft and others have incorporated them into their pipelines (which is much more revealing than a few papers showing higher bars for Neural Networks). In these scenarios, Neural nets will parametrically "memorize" the right answers for all inputs, so you don't have to keep the original data around, only the weighted connections. Anyway, I wrote a smaller (and related) rant on this topic on G+:
https://plus.google.com/100209651993563042175/posts/4FtyNBN5... |