Hacker News new | ask | show | jobs
by turmeric_root 1183 days ago
though unless you've disabled sampling it will be difficult to determine how prompts affect the output, these could just be due to RNG
1 comments

just run the 13b model 4bit quantized locally, it's already better than the 7b-8bit and you can turn down the temperature to 0 to get repeatable results.