|
|
|
|
|
|
by dchest
361 days ago
|
|
|
It's 4-bit quantized (Q4_K_M, 2.5 GB) and still works well for this task. It's amazing. I've been running various small models on this 8 GB Air since the first Llama and GPT-J, and they improved so much! macOS virtual memory works well on swapping in and out stuff to SSD. |
|
|