[riku_iki]

> The TPU is basically an ASIC as far as I know; it competes against CUDA in a very small subset of it's featureset. CUDA is essentially a composition layer on top of multiple GPU features that optimizes them for general-purpose compute.

my understanding is that compilers can compile some straighforward JAX, TF, Pytorch programs to both Cuda and TPU, so they in direct competition in current hot topics (LLM, deep learning).

[talldayo]

Right; but you can't cross-compile everything. This is really common in AI libraries, especially multi-target projects like ONNX: https://onnx.ai/

The math probably adds up in Google's favor with the TPUs, even if they end up being less efficient and slower per-unit than Nvidia hardware. They don't need to pay for the margins, and they can run them 24/7 for their intended purpose. The previous-generation TPUs can't be reused or resold for other purposes though, and if/when AI blows over as a trend you probably can't easily start mining crypto or doing HPC calculations like an Nvidia cluster would.