[lxdlam]

I have a serious question about the term "AI TOPS". I find many conflicting definitions while others say nothing. A meaningful metric should at least be well defined on its own term, like in "TOPS" or expanded "Tera Operations Per Second", what operation it will measure?

Seemingly NVIDIA is just playing number games, like wow 3352 is a huge leap compared to 1321 right? But how does it really help us in LLMs, diffusion models and so on?

[diggan]

It would be cool if something like vast.ai's "DLPerf" would become popular enough for the hardware producers to start using it too.

> DLPerf (Deep Learning Performance) - is our own scoring function. It is an approximate estimate of performance for typical deep learning tasks. Currently, DLPerf predicts performance well in terms of iters/second for a few common tasks such as training ResNet50 CNNs. For example, on these tasks, a V100 instance with a DLPerf score of 21 is roughly ~2x faster than a 1080Ti with a DLPerf of 10. [...] Although far from perfect, DLPerf is more useful for predicting performance than TFLops for most tasks.

https://vast.ai/faq#dlperf

[az226]

We don’t need this. We can easily unpack Nvidia’s marketing bullshit.

5090 is 26% higher flops than 4090, at 28% higher power draw, and 25% higher price.

[az226]

The 5090 TOPS number is with sparsity at 4bits, so it doubles the value compared to the 8bit sparse number for 4090.

The real jump is 26%, at 28% higher power draw and 25% higher price.

A dud indeed.

[lxdlam]

It really sucks. BTW, how did you find the statement? I cannot find it in any place.

[az226]

I didn’t find it. I dug up the real/raw numbers and did the math.