Hacker News new | ask | show | jobs
by sinker 41 days ago
You're right. I want to say that openai is a viable alternative, but they're even less trustworthy.

It might be time for me to start looking into Chinese models or purchasing hardware for local llms, even if the cost amounts to 5-10k.
1 comments
ai_fry_ur_brain 41 days ago
For 15k you could run the best open source models (that would require 200k in hardware to run) 24/7 at 50 tps for like 5 years straight.

Im really not sure why anyone would spend 15k to run local llms. The models you'll be able to run (70b) param models will be incredibly underwhelming.

link
choo-t 41 days ago
> For 15k you could run the best open source models (that would require 200k in hardware to run) 24/7 at 50 tps for like 5 years straight.

Is this assuming no price increase and no throttling ?

link