[romaniv]

>Before deep learning people would manually design all these extra features sin(x_1), x_1^2, etc.

It's probably worth pointing out that this is true for ANNs, but there were (and are) other "shallow" classifiers that can handle swiss roll problem without manual parameter encoding. SVMs, for example.

http://cs.stanford.edu/people/karpathy/svmjs/demo/

[conceit]

needs another image link for visualization