Y
Hacker News
new
|
ask
|
show
|
jobs
by
tommy_axle
58 days ago
Pick a decent quant (4-6KM) then use llama-fit-params and try it yourself to see if it's giving you what you need.
1 comments
gunalx
58 days ago
I habe found llama-fit sometimes just selects a way to conservative load with VRAM to spare.
link