[ranger_danger]

with regular llama.cpp on a 3070ti I get 60tok/s TG with the 9B model, it's quite impressive.