[speedgoose]

LLaMa 3.1 has been pre-trained on 15 trillion tokens, plus some more millions for the fine-tuning. About 60 terabytes.

https://github.com/meta-llama/llama-models/blob/main/models/...

The heaviest quantised LLaMa 3.1 8B is about 3.4GB.

So 0.005% compression rate, if you don't mind the intelligence of a heavily quantised 8B model.