Y
Hacker News
new
|
ask
|
show
|
jobs
by
yjftsjthsd-h
125 days ago
With only 8 GB of memory, you're going to be running a really small quant, and it's going to be slow and lower quality. But yes, it should be doable. In the worst case, find a tiny gguf and run it on CPU with llamafile.