[rjb7731]

The inference on the gradio demo seems pretty slow, about 250 seconds for a request. Maybe I am too used to the 4-bit quant version now ha!

[sebzim4500]

I'm sure it's partially the HN hug of death.