Hacker News new | ask | show | jobs
by DarthNebo 1041 days ago
JM2C

lood_in_4bit=True will let you run Llama2-7B variants at 6.3GB VRAM.