Hacker News new | ask | show | jobs
by 3abiton 81 days ago
To be fair, it's "possible" to run such setup with llama.cpp with ssd offload. It's just abysmal TG speeds. But it's possible.