Hacker News new | ask | show | jobs
by bradley13 102 days ago
I have it running locally, but speed is a problem. I have the 35GB model running on a PC with 64GB, a fairly new processor and a mid-level GPU. Ask a question, go drink a coffee.

I mean, it's great that so many models are open-source and readily available. That is hugely important. Running models locally protects your data. But speed is a problem, and likely to remain a problem for the foreseeable future.