[itissid]

Noob Q: If I have 25000$ can I buy some GPUs and support the effort for supporting GPU Poor? Like is there a DIYer's guide to do this?

I don't want to donate the cash but instead set up a handful of GPUs(Somehow?) and let people pay at cost to use them.

[redox99]

25K will either get you money for renting servers, or allow you to jerry rig yourself something like a 8x4090 machine.

The servers that most people would like to run (8xA100, 8xH100) are over 200k, and even if you had the money they're probably very hard to get right now.

[naillo]

8x4090 machines should be a lot cheaper than that. 1599*8=12,792. I.e. closer to 16x4090 machines. Sure labor is worth something but he has it in cash, and also there's barely a lot of people (yet) paying high prices for research (non ML ops) labor hours for this yet so the salary wouldn't be that high anyway.

Just a pet peeve when ppl under exaggerate how many gpus/flops a dollar can buy (if you're slighly smart with your money).

[junipertea]

What about PSU to power all of this, GPU motherboards, SSDs, CPU to not bottleneck this and sufficient RAM? You could argue it's more than 8, but definitely not 16. To utilize those GPUs to the max you need memory bandwidth to even feed it enough data.

[fyloraspit]

Yeh I built a dual 3090 machine and just to have those 2 cards run well cost a lot for all the other stuff. Workstation spec mobo, top tier PSU, CPU and memory with headroom for the system, not to mention storage and case and cooling. And this is just a hobbyist build. Cost would have easily doubled if I actually got server grade CPU and memory etc.

[redox99]

Meh, I was just ballparking. But add a threadripper, compatible motherboard, lots of RAM, fast storage, a few beast PSUs, and tax, and you're probably around 18k. So you ain't getting 2 of these machines for 25k.

Plus you'll need to set aside some money for the power bill lol

[bcatanzaro]

Also possibly upgrading the wiring in your house or office.

[capableweb]

vast.ai can help you offer this and has bunch of docs on how you can start hosting: https://vast.ai/docs/hosting/hosting

You won't be able to limit it to a particular set of clients who want to use it, but you're basically supporting the "GPU Poor" by doing this, as I think it's mostly smaller companies, individuals and researchers using vast.ai, rather than huge companies.

[rajko_rad]

honestly would recommend using spot/rented instances on that budget vs. buying! A100s run around ~$2/hour, the hard part is getting a large allocation... training workloads are quite spikey anyways, usually running lots of smaller experiments before actually doing a larger single run...

[brailsafe]

No intention to attribute malice, but wouldn't this be more like buying a condo and renting it out to pay your mortgage rather than donating resources?

[rajko_rad]

we didn't buy a condo haha, are letting people choose their own housing :)

[itissid]

Assume location is East Coast(NYC).