Because bandwidth-optimized computers do more than just matrix-multiply all day long.
Tensor Processing Units are too specialized: they can't traverse a linked list, they can't traverse trees. They're good at one thing and one thing only: matrix multiplication.
GPUs are still bandwidth-optimized and are good at matrix multiplication (but not as good as tensor units). But GPUs can traverse trees and new data-structures. Ex: BVH trees for raytracing, or linked lists... or whatever else you need. Its a general computer, a weird... terrible latency computer with HUGE bandwidth... but that's still useful in many compute applications.
--------------
Matrix multiplication is the cornerstone of many scientific problems. But you still need software to manipulate the data into the correct "form", so that the matrix multiplication units can then process the data.
Its in this "preprocessing" or "postprocessing" phase where GPUs do best. You can implement bitonic sort for highly-parallel sorting / searching. You can perform GPU-accelerated join networks for SQL. Etc. etc.
And even then, NVidia's A100 have incredibly good matrix multiplication units. So you're really not losing much anyway.
Practically, GPU is still way more generalized than TPU. For that reason, optimizing an algorithm on TPU is usually much trickier than GPUs. There are much more nuances because of the way TPU is designed.
On the other hand, Google has good reason to hold back TPUs from general public, and instead only offer them on Cloud. That also contributes its limited use.
1. Technically, TPU is less generalized, making it publicly appealing requires too much engineering effort, which has very high risk of not paying off. Case in comparison: How AMD is not able to capitialize on GPU in Deep Learning despite more uniform architecture.
2. TPUs does have some advantage, that Google want to keep at its own possession. For example, Google would not want FB to easily copy TPUs. FB indeed very likely benefit from TPUs, as they are running similar business.
Tensor Processing Units are too specialized: they can't traverse a linked list, they can't traverse trees. They're good at one thing and one thing only: matrix multiplication.
GPUs are still bandwidth-optimized and are good at matrix multiplication (but not as good as tensor units). But GPUs can traverse trees and new data-structures. Ex: BVH trees for raytracing, or linked lists... or whatever else you need. Its a general computer, a weird... terrible latency computer with HUGE bandwidth... but that's still useful in many compute applications.
--------------
Matrix multiplication is the cornerstone of many scientific problems. But you still need software to manipulate the data into the correct "form", so that the matrix multiplication units can then process the data.
Its in this "preprocessing" or "postprocessing" phase where GPUs do best. You can implement bitonic sort for highly-parallel sorting / searching. You can perform GPU-accelerated join networks for SQL. Etc. etc.
And even then, NVidia's A100 have incredibly good matrix multiplication units. So you're really not losing much anyway.