[kunstmord]

Here's a write-up about a Neural Net which was used to win a Kaggle image classification challenge, they did a lot of transformations on the input data to a) prevent overfitting b) provide invariance. Some other cool tricks mentioned there, too. https://benanne.github.io/2015/03/17/plankton.html

[benanne]

Thanks for the plug :) This is not quite the same thing though, we used a bunch of affine transformations for data augmentation, but we're not using any transforms with fancy invariance properties to compute the feature maps inside the networks, which I think is what therobot is talking about.

I have experimented with FFT convolutions (the Theano implementation for this is based on my code), but they are only really beneficial with large filters, and the current trend is towards convnets with very small filters (1x1, 2x2 or 3x3).