|
|
|
|
|
|
by SwellJoe
8 days ago
|
|
|
That explains the very old models mentioned on the page. There are models you can run locally on anything that would be more useful than llama 2 70B. e.g. the new Prism Bonsai 27B ternary model is 6GB, runs on any modern Mac, any PC with a gaming GPU bigger than 8GB (12GB or 16GB would be better), and probably runs on CPU faster than this runs any of those models. Gemma 4 E2B or E4B or 12B in the QAT quantization are fantastic for their size, and run on a lot of stuff. Models at every level have gotten a lot better since 2022. |
|
|