Hacker News new | ask | show | jobs
by shostack 1062 days ago
What are my options for running llama 2 on a single 3080?
1 comments
brucethemoose2 1062 days ago
Llama.cpp, through kobold.cpp, offloading some of it to ram.
link