[dist-epoch]

This article suggests they sort of did: 90% of the flops is in matrix multiplication units.

They leave some performance on the table, but they gain flexible compilers.