[haldujai]

Huh, this doesn't seem right. Based on #s you seem to be referring to pods but even then I'm not familiar with such a configuration existing.

A single TPUv2 chip has 1 core and 8gb of memory. A single device comes in the v2-8 configuration with 8 cores and 64gb of memory.

Pod variants come in v2-32 to v2-512 configurations.

[simonster]

A single TPUv2 host has 8 TPU cores with 64GB of total HBM (8GB per core), but like GPUs, TPUs can't directly access a network, so the host also needs CPUs and standard RAM to send data to them. They are fast, and the host has to be fast enough to keep them fed with data, so the host is pretty beefy. But FWIW, a TPUv2 host has somewhere around 330GB of RAM, not 1.4TB.

[haldujai]

Thanks for clarifying, I misinterpreted the commenter as referring to the accelerator as the conversation was about TPU availability for purchase.

I know just enough about the architecture to facilitate using TPUs for research training runs but I'm not sure what's so special about the host?

Sure it's beefy but there are much beefier servers readily available.

[simonster]

There's nothing super-special about the host. The accelerators are the special part (and, as described elsewhere, they are orders of magnitude more powerful than the Edge TPU). However, if you're an academic/independent researcher, being able to access a system with that much system memory/CPU cores for free through TPU Research Cloud is potentially appealing even without the accelerators.