Hacker News new | ask | show | jobs
by pcthrowaway 608 days ago
This is a pretty good case for just using a local model. Even if it's 50% worse than Anthropic or whatever the gap is now between open models and proprietary state of the art, it's still likely 'good enough' to categorize a story in an old newspaper as missing from an author's known bibliography.
1 comments

Good point. I use llama3.1 for a lot of small tasks and rarely feel like I need to use Claude instead. It’s fine. I’m even running the model a (big) step down from 70b, because I’ve only got 32GB of ram. It’s a solid model that probably costs me next to nothing to run.