Hacker News new | ask | show | jobs
by MacsHeadroom 1208 days ago
In int8 7B needs only 9GB of VRAM and 13B needs only 20GB on a single GPU. https://github.com/oobabooga/text-generation-webui/issues/14...