[raesene9]

Fixed cost in that if you host on a machine you know how much you'll pay a month. Obviously capping it out based on usage.

As opposed to API based where you pay more per use.

Remember this isn't a commercial service that OP is creating, it's an amusing Chatbot, they have no revenue, so fixed costs are likely to be a better model.

[baobabKoodaa]

Local LLM won't give you "fixed costs" any more than using an API. If they want to serve the users of their amusing chatbot, they will have to scale up their local LLM according to scale, and they won't have "fixed costs" anymore. If they don't want to scale up and they're fine denying service to users at peak hours, they can do that with an API too. There's no law that says if you use an API then you must allow unlimited use. Of course they can also limit the use of the API.