Google is already there with TPUs. The reason they can add AI to every single google search is not just that Google has near-infinite cash, but also that inference costs far less for Google than anyone else.
Reading the specs on the new TPU designs and how it incorporates optical switching fabric and other DC-level technology to even function, I think the moat is already there.