|
|
|
|
|
|
by havfo
46 days ago
|
|
|
I was able to solve this for my setup, 7900XTX and llama.cpp on ROCM in the oh-my-pi fork of pi.dev harness. I documented my setup on github, check under my username/omp-config, but the important thing is making sure the context is strictly append-only, and starting llama.cpp with --chat-template-kwargs '{"preserve_thinking":true}'
|
|
|