|
|
|
|
|
|
by piskov
753 days ago
|
|
|
You’ll need 44GB just for the weights By default only 75% of unified memory is available to GPU if you have >36GB. So with 48 total only 36 is available for GPU with is lower than 44. tldr; without quantization you will not be able to run it. |
|
|
This article says 88GB without quantization. Though it then goes on to make a ridiculous claim that if you had 128GB of RAM, then using up 88GB of 128GB would make everything else really slow because I guess the think the remaining 40GB of RAM somehow isn't enough for your OS and desktop apps.
So it’s probably not a very authoritative source.