[woadwarrior01]

Interesting! This also seems to work with smaller quantised models. I just tried it with a 4-bit quantised version of WizardLM 13B v1.2 and it seems to work quite well.