Analog circuits run at full-speed (no clock), use little power, and take up little space. An analog computer directly implements the mathematical function it represents as circuits instead of emulating it on a von Neumann-like machine. Long story short, they have issues that made people go digital. Yet, if you can use analog, you can get significant advantages. Example for math acceleration:
Brain is a bunch of components that are spread out 3D that operate like a mathematical function at slow speed. Mostly sounds analog. Results in us. So, a huge spread of analog components could get some results directly simulating something like that. Here's one of my favorites which is a wafer-scale, analog computer for neural networks.
If you have an application that can tolerate error (like classification), then analog computing can give enormous gains in terms of speed _and_ power efficiency. Essentially, the savings come from using physics to perform the math (see Kirchhoff's current law) vs. using discrete time steps vs. fully-unrolling the logic. Google may not be using analog processing for this version, but I read an analog neural network researcher's page who said he moved to Google last year. (Sorry, I can't find the page again, but I think he was from the UK.)
True. ML applications are well-suited to analog computation not only because they can tolerate errors, they also have an ability to adapt to errors, provided training algorithms are ran in hardware.
For the curious, Optalysys has built a general purpose optics-based correlation/pattern matching machine. From some of their predecessor-company marketing material: The correlator performs pattern matching on large data sets such as high-resolution images, providing a measure of similarity and relative position between objects within the input scene. This allows large images [and general data converted to images] to be analysed far faster than electronic equivalents.
Going back to the topic of NN-based computing, I found this talk to be intriguing: https://www.youtube.com/watch?v=dkIuIIp6bl0. The main argument is that because Moore's law may no longer be in effect, it will become increasingly important to explore alternate computing solutions. (Google's TPU could be supporting evidence for this argument.) The speaker also co-authored a paper which I liked "General-Purpose Code Acceleration with Limited-Precision Analog Computation".
This video was very cool. Are there any IC's that can perform analog computing for neural networks on the market now? I'm picturing something like an FPGA but with a bunch of op amps that you can connect into summers or amplifiers.
If not, how would one practically implement an analog computer for neural network programming (without several tables full of op-amps?)
You can implement an analog neural network yourself using a Field Programmable Analog Array. (I've never done it, but you'll see academics online writing papers about it.)
Another thing that is sort of related is Lyric Semiconductor; they built these cool application-specific probabilistic processors; they were purchased by Analog Devices a while back.
http://www.cisl.columbia.edu/grads/gcowan/vlsianalog.pdf
Brain is a bunch of components that are spread out 3D that operate like a mathematical function at slow speed. Mostly sounds analog. Results in us. So, a huge spread of analog components could get some results directly simulating something like that. Here's one of my favorites which is a wafer-scale, analog computer for neural networks.
www.kip.uni-heidelberg.de/Veroeffentlichungen/download.cgi/4713/ps/1856.pdf