Hacker News new | ask | show | jobs
by cerol 1188 days ago
Can someone provide a guide on how run LLaMA on a fairly average CPU/Nvidia GPU?
2 comments
eulers_secret 1188 days ago
Another great option is https://github.com/oobabooga/text-generation-webui

The 7B model will run without changes on a 3080. The 13B 4-bit model also runs on a 3080.

This Reddit post has the instructions I followed: https://old.reddit.com/r/LocalLLaMA/comments/11o6o3f/how_to_...

link
isoprophlex 1188 days ago
Check the git repo for llama.cpp.

Model weights in torrent magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352

Runs slowly but without trouble on an intel i7 with 32 gb ram.

link