[verdverm]

Given the TPUv5 improves perf/$$$, it would seem to be at odds with your comment. I can now get more done with the same spend.

Kelsey Hightower told me at a GopherCon (many years ago) that Google doesn't run any internal workloads on third-party GPUs mainly because it costs significantly more (b/c cooling iirc), though they are happy to help you run your workloads on such GPUs.

[dekhn]

Google runs internal workloads on third-party GPUs (especially as TPUs weren't very good at sparse for a long time). Hightower was simply wrong.

[holografix]

Hightower’s biggest skill is sounding very knowledgeable and as if he has the keys to a gate of useful secrets.

[verdverm]

Kelsey is kind, thoughtful, and never puts another person down, I would say empathy is his best skill.

He also shares what he has learned for free, rather than putting paywalls in front of his content, which is quite rare these days.

[webmaven]

Which (potentially) draws compute spend from Microsoft Azure to Google Cloud.

[verdverm]

I should have quoted the generalized middle statement I was responding to

> AI is all about how much compute dollars it can generate for the cloud providers.

If the providers wanted to extract more money they would not create custom hardware which reduces overall costs and prices to users.

I would argue that this is actually more about ensuring NVidia doesn't have a monopoly on hardware and alleviates us from having to pay for Nvidia profits through our cloud providers.

Azure is going down the same path here: https://www.theverge.com/2023/11/15/23960345/microsoft-cpu-g...

[qeternity]

> If the providers wanted to extract more money they would not create custom hardware which reduces overall costs and prices to users.

Extracting money is about margins, not revenues. If they reduce your costs (and their revenue) by 20% with a TPU, but they can produce TPUs for 50% less than buying gear from Nvidia, it's still a profitable move.

[verdverm]

Exactly, if I end up paying less and the cloud also makes more money, seems like a win for everyone

The "extracting" word typically comes with abusive connotations when used in the context of money, which doesn't feel like the right word for the win-win outcomes imho

[freen]

Jevons paradox.

Higher efficiency results in greater utilization.