[roynasser]

A common model in some cost cutting software is to charge x% of the total savings... Win/win...just a suggestion... use picks "main LLM" and you calculate the "non optimized cost" based on that. Whatever savings you drive you take a share of the savings.

[janekm]

It's tough in this case, because if you incentivise just to save cost, it could always route you to the cheapest LLM but the quality would suffer...

[danlenton]

however, as janekm says, we can't charge just based on cost savings. We would need the router points to be sufficiently compelling wrt quality, speed and cost (including our own margins) that users still sometimes opt for these router points. Suffice it to say, if any router configs do start to take margins, then this will be clearly reflected in the overall router cost plotted on the scatter graph. UX will not be affected.

[danlenton]

Yeah that's a great point, something we'll keep in mind as we work out the final business model. Thanks!