[shishirpatil]

Yes indeed. The models are too computationally expensive to run locally (7.5Billion parameters). Though you could in-principle swap in any local model.

[kajecounterhack]

Do y'all have plans to release the model for those who have 16gb graphics cards? (I'm assuming the model is fp16?)

[Tostino]

What are you talking about? 7b parameter models run insanely fast if you can offload to gpu, and are entirely reasonable speed if CPU only.