Hacker News new | ask | show | jobs
by Mashimo 3 days ago
Why would you use this when your company has access to actual SOTA? I don't get it.
1 comments

Why would I ever use a local model by that logic? A usable model means my computer was very expensive so I‘d have the funds for a Pro plan as well.
Well we are on HACKER news ;-) To mess around and learn something would be one reason. Maybe you already have the hardware. Why selfhost anything if cloud does the thing?

An other reason would be because your company does not allow any source code leaks and thus every developer either has local models or none.

6 T/second local model while your colleagues have 200 EUR/month claude does not make much sense. At least I can't see a use case.