[thu2111]

That's probably a part of it. Note how the Google TPUs aren't for sale. If you want them you have to use the Google Cloud. The cloud is expensive and slow ... I think everyone is shocked when they first see perf numbers coming off Azure.

I don't know if GCE is better, but the temptation to overload the hardware is always there: hardware rental is fundamentally a business with low barriers to entry. Anyone can buy some machines, bring up a Kubernetes or OpenShift cluster and start renting it out. So the big 3 are always looking for proprietary advantage and dedicated AI chips are something other firms can't easily do at the moment, making it a good source of lockin.

Do many people need it though? Deep learning is pretty useless for most business apps, unless you happen to need an image classifier or something else pre-canned. Classical ML is often sufficient, or better, human written logic. The latter can be explained, debugged, rapidly improved and in the best case requires no training data at all!

[dijit]

I work for a company that uses bare metal, open stack, azure, AWS, and GCE.

GCE perf is significantly better (and more consistent) than Azure. Even with Windows instances. :/

But I agree that “cloud is slow” when compared to bare metal- it’s also most financially costly especially for the same performance due to it being slower. But the gains in flexibility are immeasurable.

[TrinaryWorksToo]

Their tpus are for sale: https://coral.ai/products/

[raverbashing]

These are Edge TPUs, for inference, not for training

[The_rationalist]

I thought a single powerful gpu would be enough for any inference?