[noosphr]

Because llm inference is not the only workload a GPU can do and custom silicon cost $10b a chip.

[eru]

AI optimised cards is the biggest cashcow for nvidia. They are definitely doing custom silicon. And Google et al have cards that don't even pretend to be able to do graphics.

[noosphr]

There is more than one workload in AI. Inference for llms is memory constrained on even a single card. Training for llms is memory constrained on the level of racks. In both cases you hardly ever see more than 40% of advertised flops used.

[eru]

So the question is: why are they designing AI cards so unbalanced?