[bitL]

I think at least for inference TPUv1 was beating all previously available GPUs by a wide margin. TPUv2 did that for training as well, with the exception of Volta.