| Based the following constraints that lie at the center of AI and parallelism, I'd say no -- stochastic gradient pursuit using vector processors like GPUs is inescapable in all future AI advances. 1) All AI is based in search (esp. non-convex, where heuristics are insufficient to provide a global convex solution), and thus is inevitably implemented using iteration, driven locally by gradient-pursuit and globally by... ways to efficiently gather information to optimize the loss function that measures how well that info gain is being refined and exploited. 2) Search that is inherently non-convex and inefficient requires as much compute power as possible, i.e. using supercomputers. 3) All supercomputer-based solutions to non-convex problems are implemented iteratively, where results are improved not using closed-form math or complete info, but by incremental optimization of partial results that aggregate with the iterations, like repeated stochastic gradient descent that creates and enhances 'resonant' clusters of 'neurons'. 4) The only form of supercomputing that has proven to scale up at anywhere near indefinitely is data-parallelism (a dataflow-specific form of SIMD) -- where the search space is spread as evenly (and naively) as possible across as many processing elements as possible. 5) Vector processing hardware like GPUs implement data-parallelism as well as any HPC architecture yet devised. Thus, I believe that AI is stuck with GPUs, or equivalent meshes of vector processors, indefinitely. |
Quantum computers are not magic and they are by no means general-purpose, but if your search mechanism matches the kind of search quantum computers are good at, they can greatly exceed the speed of GPUs.