[forrestthewoods]

What type of system is needed to self host this? How much would it cost?

[fragmede]

One GB200 NVL72 from Nvidia would do it. $2-3 million, or so. If you're a corporation, say Walmart or PayPal, that's not out of the question.

If you want to go budget corporate, 7 x H200 is just barely going to run it, but all in, $300k ought to do it.

[gloflo]

How many users can you serve with that?

[fragmede]

For the H200, between 150-700. The GB200 gets you something like 2-10k users.

[forrestthewoods]

Whoa. How on earth can one system serve 2000 potentially concurrent users?

[disiplus]

Depends how many users you have and what is "production grade" for you but like 500k gets you a 8x B200 machine.

[p1esk]

Depends on fast you want it to be. I’m guessing a couple of $10k mac studio boxes could run it, but probably not fast enough to enjoy using it.

[CamperBob2]

$20K worth of RTX 6000 Blackwell cards should let you run the Flash version of the model.