GPUs and multicore CPUs do not perform the same work as the single core CPUs that dominated before ~2003, when Moore's Law first slowed. By expanding the measure of speedup to include GPU/multicore, arguments like yours require not only a change in hardware but in benchmark code as well.
Longstanding general app benchmarks like SPEC emphasized everyday tasks that rarely benefit from parallelism, appropriately revealing the general ineffectiveness of adding GPUs or multicore to everyday apps. The only fair way to assess the impact of GPU/multicore is to continue using the same benchmark code as when mono-core CPUs reigned. When you do that, the value of adding GPU/multicore essentially disappears and the speedup of Moore's Law duly fades (again ~2003).
Thus until users begin to run deep learning code on their computer's GPU, AI-code won't speed up exponentially, nor continue with future GPU advancement. The hardware basis driving Kurzweil's Singularity has truly run out of steam.
GPUs and multicore CPUs do not perform the same work as the single core CPUs that dominated before ~2003, when Moore's Law first slowed. By expanding the measure of speedup to include GPU/multicore, arguments like yours require not only a change in hardware but in benchmark code as well.
Longstanding general app benchmarks like SPEC emphasized everyday tasks that rarely benefit from parallelism, appropriately revealing the general ineffectiveness of adding GPUs or multicore to everyday apps. The only fair way to assess the impact of GPU/multicore is to continue using the same benchmark code as when mono-core CPUs reigned. When you do that, the value of adding GPU/multicore essentially disappears and the speedup of Moore's Law duly fades (again ~2003).
Thus until users begin to run deep learning code on their computer's GPU, AI-code won't speed up exponentially, nor continue with future GPU advancement. The hardware basis driving Kurzweil's Singularity has truly run out of steam.