[F7F7F7]

The problem with tools like this is that somewhere in the chain between you and the LLM are token reducing “features”. Whether it’s the system prompt, a cheaper LLM middleman, or some other cost saving measure.

You’ll never know what that something is. For me, I can’t help but think that I’m getting an inferior service.

[revnode]

You can self host something like https://big-agi.com/ and grab your own keys from various providers. You end up with the above, but without the pitfalls you mentioned.

[mark_l_watson]

BIG-AI does look cool, and supports a different use case. ABACUS.AI takes your $10/month and gives you credits that go towards their costs of using OpenAI, Anthropic, Gemini, etc. Use of smaller open models use very few credits.

The also support an application development framework that looks interesting but I have never used it.

[mark_l_watson]

You might be correct about cost savings techniques in their processing pipeline. But they also add functionality: they bake web search into all models which is convenient. I have no affiliation with ABACUS.AI, I am just a happy customer. They currently let me play with 25 models.

[freedomben]

If anyone from Kagi is on, I'd love to know, does Kagi do that?