|
|
|
|
|
|
by hbbio
1202 days ago
|
|
|
Happened to me as well.
Apparently, you can just run: python3.10 convert-pth-to-ggml.py models/7B 1
./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2
And then play with: ./main -m ./models/7B/ggml-model-q4_0.bin -t 8 -n 128 -p "..."
|
|
|