[MasterScrat]

Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work.

I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!

[memossy]

The v5es and v5ps are pretty amazing at running SD, giving code for SD3 now to optimise it on those.

v5es are particularly interesting given the millions that will land and the large pod sizes, particularly well constructed for million token context windows.

[doctorpangloss]

But you and I can't buy a TPU. You and I can buy an H100.

[isoprophlex]

Speak for yourself! I can't even afford 1/10th of an H100.

[elorant]

He's probably speaking about availability, not affordability.

[leansensei]

It was an attempt at humor, obviously.

[renewiltord]

Which TPUs do you use? Cloud-hosted or your own hardware? Interesting insight.

[1024core]

"TPUs" are a Google-only product, available* only on GCP.

* Notwithstanding the Choral boards

[MasterScrat]

We started on V3s, now fully moved to V4s with some V5Es, investigating a full move towards V5E & V5P

[memossy]

We use v4s, v5es & v5ps. Mostly v5ps, very stable int8 training (versus the horror that is fp8 stability)