Hacker News new | ask | show | jobs
by kylehotchkiss 16 days ago
Where are the computers we could purchase to run 124b models :’(
1 comments

You can get SXM V100s for like $100 off ebay, if you're willing to do the troubleshooting work to get em running with adapters you can build a computer capable of fitting a Q4 quant of a 120b model in VRAM for something like fifteen hundred dollars. (assuming you already have some RAM sticks laying around T___T)