|
|
|
|
|
|
by ashertrockman
1884 days ago
|
|
|
The choice of activation function isn't entirely clear to me, but I think it's definitely possible to make a network that operates entirely in the frequency domain. It would probably be pretty easy to start experimenting with such a thing with the nice complex number and FFT support in PyTorch 1.8. :) Like you said, there's already a significant connection between convolutional networks and the Fourier domain (the convolution theorem). Tangentially, I've recently worked on a project that focused on implementing convolution in the Fourier domain, and how that allows one to control useful properties of convolutions (like "smoothness" and orthogonality). I made a demonstration of convolution in the Fourier domain in PyTorch, which you might find interesting: https://nbviewer.jupyter.org/github/locuslab/orthogonal-conv... More generally, you could look here for more code and the corresponding paper: https://github.com/locuslab/orthogonal-convolutions |
|
|