[chimtim]

All this raw compute/parallelism is great but it does not really help the algorithm output in terms of efficiency. The CPU/GPU difference is much smaller than nvidia would like you to believe especially for more complex networks which are becoming more and more common now. Of course if you want to just do convolutions (which the TPU paper claims is only 5% of their google workload), building a hardware around it may work well for specific algorithms.