[yxchng]

How big is the dataset you usually deal with? From my experience, you hit computational bottleneck pretty quickly with GPs (100k datapoints of maybe 100 dimensions is pretty much the max you can deal with).

And if you are talking about dataset of this scale, then I agree with you that GPs are better than NN. However, people are excited about NN capability of dealing with immensely huge and high dimensional dataset not small scale ones.

[cultus]

Thousands to millions. There are approximations that work very well with millions of examples.

Neural networks are empirically outperformed by gradient boosted trees (look at Kaggle competitions) on most practical tasks except for image, sound, and video problems.

Neural networks can be very slow on large datasets. Training can often take days or weeks, even with a GPU. GBTs and GP approximations are faster.