Hacker News new | ask | show | jobs
Show HN: Run Llama3.1 405B on a 8GB VRAM challenge [video] (youtube.com)
19 points by lyogavin 689 days ago
How to run Llama3.1 405B on a 8GB VRAM
1 comments
langcss 688 days ago
https://github.com/lyogavin/airllm

What sort of speed do you get?

link
lyogavin 688 days ago
Not fast, not a good fit for realtime interaction applications, but for offline data processing cases, it works perfectly. I use it for my offline tasks.
link