| > The startup costs for just messing around at home are huge No, they are zero. Most people have extra hardware lying around at home they're not using. It costs nothing but time to install python. $100 is not free. If you can't be bothered, sure thing, slap down that credit card and spend your $100. ...but, maybe not so for some people? Consider students with no credit card, etc; there are a lot of people with a lot of free time and not a lot of money. Even if you don't want to use it do you do seriously think this project is totally valueless for everyone? Maybe, it's not for you. Not everything has to be for everyone. You are, maybe, just not the target audience here? |
The difference between an open model running on a $100 computer and the output from GPT4 or Claude Sonnet is huge.
I use local and cloud models. The difference in productivity and accuracy between what I can run locally and what I can get for under $100 of API calls per month is huge once you get past basic playing around with chat. It’s not even close right now.
So I think actually you are not the target audience for what the parent comments are taking about. If you don’t need cutting edge performance then it’s fun to play with local, open, small models. If the goal is to actually use LLMs for productivity in one way or another, spending money on the cloud providers is a far better investment.
Exceptions of course for anything that is privacy-sensitive, but you’re still sacrificing quality by using local models. It’s not really up for debate that the large hosted models are better than what you’d get from running a 7B open model locally.