Hacker News new | ask | show | jobs
by wxw 6 days ago
Hm, looks like this is an old project (2022) associated with HuggingFace (https://huggingface.co/bigscience).

Doesn't look like it's very active nowadays: https://github.com/bigscience-workshop/petals

Article with more details: https://techcrunch.com/2022/12/20/petals-is-creating-a-free-...

> [...] volunteers can donate their hardware power to tackle a portion of a text-generating workload and team up others to complete larger tasks, similar to Folding@home and other distributed compute setups.

2 comments

That explains the very old models mentioned on the page. There are models you can run locally on anything that would be more useful than llama 2 70B. e.g. the new Prism Bonsai 27B ternary model is 6GB, runs on any modern Mac, any PC with a gaming GPU bigger than 8GB (12GB or 16GB would be better), and probably runs on CPU faster than this runs any of those models. Gemma 4 E2B or E4B or 12B in the QAT quantization are fantastic for their size, and run on a lot of stuff. Models at every level have gotten a lot better since 2022.
Maybe it's time to refresh this idea then. There's a clear demand for running LLMs at home, but the good ones are too big, and you're not using it most of the time. Sharing with others sounds like a perfect solution to me.