[scheme271]

We don't know the parameters but it probably takes at least a H100 and possibly several to run a SOTA model. Given the pricing (25+k per H100 + hardware to run it) and power (700W per H100 + hardware to run it), I don't see how anyone except for a largish company can afford to run this.

[sshumaker]

Are you serious? It’s multiple nodes to run a frontier model (a node is 8x GPUs), and they aren’t running on H100s. You are looking at 32+ GPUs.

[scheme271]

I was being pretty generous to the comment I was replying to. Needing 32+ H100s just strengthens my argument that people aren't going to run frontier models locally anytime soon.