> 270M, 450M, 1.1B and 3B parameters
Which roughly translates to 3GB for the highest end one, depending on context length used.
8 bits = 1 byte
3 billion * 1 byte = 3 gigabyte
+ Some memory for the context of the LLM
[0]
3b-instruct has a total file size of 4.94GB + 1.13GB which is 6.07GB which can be seen here:
https://huggingface.co/apple/OpenELM-3B-Instruct/tree/main
A bit of overhead will always be there, as you probably want to store some metadata next to the raw weights.
> 270M, 450M, 1.1B and 3B parameters
Which roughly translates to 3GB for the highest end one, depending on context length used.