|
|
|
|
|
|
by eli_gottlieb
3603 days ago
|
|
|
>Convolutional layers have been around longer than that and don't have much to do with the vanishing gradient problem, which was never that big a problem to begin with. ReLUs are helpful, but they do not lead to THAT big of a speedup. Convolutional layers have definitely had a big impact for accuracy in image problems and sound problems. They're essentially an informed prior on image and sound problems, which vastly reduces the size of the parameter space and thus makes both vanishing gradient and overfitting less of a problem. |
|
|