[LoganDark]

It looks like the practical limit for that on my hardware is ~130 t/s, which is just okay. Makes reasoning mode quite painful though.

(llama.cpp can't seem to exceed 30 t/s, I had Fable make me custom inference. that's why it took me a day to get back to you)

GPT-3.5 would repeat my prompt back in its own words for me to confirm. This model immediately runs with it and generates a lot of output. I would definitely need to use a harness that allows editing / cutting off generations.

Something I've noticed about models that have been specifically uncensored or trained to generate content is that they aren't very engaging to chat with. Maybe this is just because they weren't trained to chat at all (base models) but, it's hard to find a chat model that's been trained properly.