[sgt101]

I think you are out of date. Rectification is pretty good at removing a lot of the vanishing gradient issues that nn's used to face, and the overwhelming power of modern digital computers (50k cores is common) make this all moot as far as I can see.

[adamnemecek]

Nope, they aren't even in the same category. E.g. Notice that there is a cpu size limit because of heat. Do you know what doesn't overheat? A photonic computer. You can build a cpu the size of a house. Also how does the number of cores constitute "overwhelming power".

[sgt101]

:o) well, due to the wavelength of light a photonic cpu has to be 50->100 times the size of a current gen electronic one.

When I were a young 'un we had one core, and it ran at 25Mhz and about 130 of us shared it. Now I have 50,000 cores that run at 2 Ghz and five people share it. Things aren't quite directly comparable but the speed up is at least 100,000x I am overwhelmed by this, things that would have taken 1000 days; approximately 3 years, can be achieved in ten or twenty minutes. In reality the use of these infrastructures has enabled (in neural net land) the development of techniques that improve performance by several more orders of magnitude - so things that would have taken several years are now done in a minute or so. I believe that there is plenty more headroom to be had.

[adamnemecek]

Where is that estimate coming from? I mean year there has been progress by why do you think photonic would be even better?

[sgt101]

1989->2017

Photonics are promising QC technologies - especially Phonons, but we are a long way off!