[zopper]

Do you run Vicuna locally or have you had any success in deploying it somewhere? I have been thinking of using a fine-tuned model but haven't found anything cost effective where I cab deploy it. You either run the machine where it's served consistently and the cost is in the hundreds or you accept extremely slow starts.

[dserban]

I have a monster of a Linux cluster at home, which is busy running Cassandra and Spark for my other side projects. I'm piggybacking on that while this particular project is still in development.

At the rate LLMs are evolving, a reasonably priced cloud offering will probably exist for me to leverage for production, by the time I need it.