[wyldfire]

How much detail do we know about the TPUs' design? Does Google disclose a block-diagram level? ISA details? Do they release a toolchain for low-level programming or only higher-level functions like TensorFlow?

EDIT: I found [1] which describes "tensor cores", "vector/matrix units" and HBM interfaces. The design sounds similar in concept to GPUs. Maybe they don't have or need interpolation hw or other GPU features?

[1] https://cloud.google.com/tpu/docs/system-architecture

[jacksmith21006]

Great paper on the Generation 1 TPU. But Google has not shared much details on gen 2 and in some ways kind of hid information.

Suspect we will need a gen 3 to get a paper on the gen 2.

Here is the gen 1 paper and highly recommend. Pretty interesting using 65536 very simple cores.

https://arxiv.org/ftp/arxiv/papers/1704/1704.04760.pdf

[elmarhaussmann]

So far only very few details are disclosed. Here are two presentations:

https://supercomputersfordl2017.github.io/Presentations/Imag... http://learningsys.org/nips17/assets/slides/dean-nips17.pdf

For the last version of the TPU, Google provided more detail, e.g., in this paper:

https://arxiv.org/pdf/1704.04760.pdf

Hopefully, Google will publish something similar for TPUv2, but I have no knowledge whether or when that might happen.

[ndesaulniers]

> Maybe they don't have or need interpolation hw or other GPU features?

Definitely, no need to do any kind of rasterization here.