Hacker News new | ask | show | jobs
by aidenn0 992 days ago
I have yet to get any useful output out of the Q5_K_S version; haven't tried any others yet.
2 comments
poser-boy 991 days ago
Linked is the base model. What you want is the instruct model (also on TheBloke's profile), which has been trained on following instructions.
link
daakus 992 days ago
I used mistral-7b-v0.1.Q5_K_M.gguf and it responded to basic questions.
link