See I absolutely dislike this thought that hyperscalars can easily beat nvidia. It is not their domain of expertise. Tpu are not where near GPU in performance. People really underestimate Nvidia's expertise and strengths.
They don't need to beat them on performance though. If you get half the performance at third the price you can just make more chips and be fine. It's not like Google is gonna run out of datacenter space.
All of this may very well be true — but it doesn't matter. Google is getting results very similar to the rest of the pack, probably at a fraction of the cost. The implementation doesn't matter so long as you get the results.