Hacker News new | ask | show | jobs
by yosito 889 days ago
How much is it costing you to run the LLM? Is it OpenAI?
1 comments

Mixtral on Anyscale Endpoints: https://docs.endpoints.anyscale.com/pricing
Thanks! I'm not experienced enough in this domain to have an idea of what that means for the bottom line. I assume that at the scale of a niche app, the cost is close to negligible and you can pay it out of pocket, but I imagine that if Amazon would pre-install something like this on every Kindle, the costs would go through the roof. Speaking in broad terms, how much scale do you think an app like this could handle, while still being something an individual might consider worthwhile to pay for out of pocket?