[georgeven]

I have a 1500 dollar machine that can run it at 50 tok/s (3 V100s)

[Dig1t]

How did you buy 3 V100's for $1500??

[sixdimensional]

Not OP and just guessing, but probably SXM2 GPU modules for the V100. Those can be acquired fairly inexpensively, but there is work to do to get them working together and the V100 has some limitations on the types of models you can run.

[georgeven]

PCIE 16 gig is <300 dollars these days a piece

[georgeven]

on eBay you can buy 4 of them for 1000. PCIE 16 gigs.