[agibsonccc]

I'd also add here that you can add other variables in to the mix such as gaussian noise and drop out which is the basis for a lot of fundamental neural networks. I get the intent, but it's not necessarily the case.

Neural word embeddings are one of the most fun things I work with. Both word2vec as well as glove and paragraph vectors.

There's also the ability to learn varying length windows of phrases via recursive or convolutional methods.