[apatheticonion]

I love the Chinese models.

I use DeepSeek exclusively and now Kimi K3 offers a great planning assistant for more advanced coding tasks.

DeepSeek v4 Flash is extremely fast and is able to handle pretty much anything I've thrown at it (I use mostly Rust, PSQL, Angular and Terraform).

I self host Bifrost as my LLM gateway, though I wish LLM vendors would do monthly/daily automatic billing (like VPS providers do) rather than prepaid + auto-top up.

It's annoying maintaining a non-refundable minimum balance across vendors, I would rather be billed for my exact usage.

OpenRouter helps, but I don't really like it as a service and not a fan of the mark up.

[arikrahman]

The best part is using harnesses like reasonix or whale make cache hit at a rate close to 98%, making requests converge to practically free. And that's with unsubsidized American providers like cloudflare or Digital Ocean.

[KronisLV]

In long form tasks, across multiple harnesses (Claude Code, OpenCode, Kimi Code, ZCode) my cache rates are typically 96-99%.

I don’t think that’s particularly out of the ordinary. Do people have different experiences with other harnesses? Which ones?

[steve-atx-7600]

How much time do you spend on your setup vs getting a lot of stuff shipped by paying for fabel 5? For me, not using the best model is a huge opportunity cost since my company can afford it.

[re-thc]

> For me, not using the best model is a huge opportunity cost since my company can afford I

But sometimes not using the "best model" is using the best model. Fable isn't the best at everything. Specialization and optimizations might not just be around cost.

[steve-atx-7600]

I can see that. I’m just afraid to sync too much time into complex routing schemes when I get pretty consistently good results out of got 5.6 or fabel. For code reviews, I’ll try the best flash and grok at the time but they just don’t come close to gpt 5.6 which has been the best review model for me since 5.5.

[KronisLV]

> How much time do you spend on your setup vs getting a lot of stuff shipped by paying for Fable 5?

Relatively speaking, quite little (and it's more interesting than some of the stuff I'm otherwise shipping), I mostly just explored to see what's available.

I also agree that non-SOTA models are risky, that's why I was shopping around for them as well - DeepSeek V4 Pro is cheap but unreliable, GLM 5.2 is around and maybe slightly past Sonnet quality (though their quotas are a bit of a problem), whereas Kimi K3 is a proper contender.

I did write a tool to manage 3rd party providers for Claude Code: https://ccode.kronis.dev/

However, in the end I figured out that for the terminal use cases OpenCode is really comfy (provider TUI solutions are okay).

For web/desktop based stuff I like the UI of Claude Code Desktop, though ZCode comes close too (which is surprising, they sorta came out of nowhere and are patching the thing weekly), meanwhile Kimi Code and OpenCode desktop/web offerings still aren't great, but are functional.

I actually did write a bit more about my experiences on my blog.

GLM 5.2 with their harness and coding plan: https://blog.kronis.dev/blog/z-ai-s-glm-5-2-is-a-great-model...

Kimi K3 with their Allegro 100 USD plan and harness: https://blog.kronis.dev/blog/kimi-k3-is-out-is-anthropic-don...

That exploration let me switch over to Kimi fully because the tone of Anthropic's models is insufferable: https://blog.kronis.dev/blog/ai-slop-is-a-self-inflicted-tra...

Not to spam too much, but maybe those experiences are useful to someone. Long story short, most harnesses are okay but shopping around a little bit is definitely a good idea, the same way how spending some time choosing a font isn't a bad thing if you'll stare at it for 8 hours a day. I'm also happy that I managed to find software that's okay to run and also a model whose tone I actually enjoy, that is still near-SOTA in performance and that I can throw tasks at it without worrying about whether the model is or isn't good enough at those.

I will admit that I'm probably slightly overspending by moving over fully to Kimi, there's probably a plateau for each kind of task and not everything needs SOTA models, but at least this way I don't have to think much about it.

[mlazos]

How can you be hitting cache on what I think are novel LLM prompts …

[oceansweep]

Not them but my understanding is that the harness will send a simple 'heartbeat' message to keep the cache 'warm', (see prefix caching: https://handbook.modular.com/inference-optimization/prefix-c... ) which can then be edited/changed, which does cause the user to incur a fee, but its much less than the amount they'd pay on a no-cache hit request.

[slopinthebag]

The prompt is only novel the first time it’s sent. Then as it’s sent repeatedly as part of the previous context it’s cached.

Deepseek afaik has a novel architecture that is somewhat forgiving of cache shifts. I’ve been getting +90% cache hits with Zed’s agent and it’s not doing anything special regarding caching afaik.

[verdverm]

multi turn sessions, they are typically in the high 90% hit rate across all providers without doing much of anything

[sghiassy]

I’ve never used either of those tools. Do they work well?

[Rzor]

Can you expand more on how you are using it?

[dools]

Yeah deepseek is my workhorse of choice these days, I mostly use pro because I do a lot of concurrent jobs so speed is less of a concern. When it falls over I change the model mid-chat to GPT5.5 and chuck a couple of tokens over to OpenAI and then once it's correctly found the problem I switch back to deepseek, keeping all the 5.5 analysis in context.

I have been a heavy user of k2.5/6 but they seem to have gotten slower and worse at their jobs (and the "engine overloaded" errors have increased ... ), and k3 spends a LOT of time thinking and doesn't produce noticeably better results. I do keep chucking some tasks over to moonshot every now and again to give it a chance. I figure they're under compute pressure and have probably quantised the models to cope temporarily.

But recently I've found myself mostly using deepseek v4 pro and gpt 5.5 when I need the big guns.

GLM 5.2 has been good at select tasks but really shit at others and given how sparingly I use GPT 5.5 I'm not that compelled to use it. I've yet to be impressed by any Google or Anthropic models!!

[MrBuddyCasino]

> GLM 5.2 has been good at select tasks but really shit at others

Yeah it really likes to shit the bed after thinking forever. Even if it can do some things, the quality is not quite up there in my experience.

[skeledrew]

Prepaid means someone can't sign up, use a bunch of inference, then cancel their card and disappear into the sunset. VPS is a more long term investment where it's harder to switch and it doesn't cost the provider much if a few users jump out without paying for a month.

[apatheticonion]

I do understand the reasoning for it, it still sucks as a user who is doing the right thing.

[equinumerous]

As someone who is also eyeing Bifrost, I am curious to know what made you choose Bifrost at all / why you decided to use an LLM gateway. I also was considering OpenRouter, since it provides pretty much every model, with same-day releases for new models. One thing that is attractive to me about Bifrost is that if I decide to leave OpenRouter tomorrow, I can do so without touching any other part of my stack; I'm hoping self-hosting models becomes more viable, and then I can become less dependent on third party LLM providers like OpenAI/Anthropic/OpenRouter, and I would not need to worry about a model that I depend on suddenly being deprecated.

[copperx]

What don't you like about the service, besides the mark up?

[apatheticonion]

For me, the only utility OpenRouter gives me is billing consolidation - I don't really need the routing capabilities because I use Bifrost for that.

As a router, it's not very feature rich. For example I restricted the available models to the ones I want to use however the `/models` endpoint still lists all the models, making my LLM client list the 200+ models available on the service (even though they will throw an error if I try to use them).

With Bifrost, I can also create model aliases with custom configuration - for example I can create a model alias `deepseek-v4-flash-nothink` which disables thinking. I can create `deepseek-v4-flash-caveman` which injects the caveman skill (to save tokens) etc.

Plus I can contribute to Bifrost, which I can't do with OpenRouter.

[robbiet480]

Did you look at LiteLLM at all? It seems fine but Bifrost looks interesting too.

[apatheticonion]

Yeah I looked at it, LiteLLM is functionally more mature.

The only reason I didn't go for it is I'm not a fan of Python dependency management, Bifrost is just a single executable that uses nearly no memory and is lightning fast.

[CuriouslyC]

LiteLLM is bad. Shit performance, buggy, and none of it surprising if you look at the tangled mess that is their codebase.

[aftbit]

Can you configure Bifrost using entirely config files without the web interface? Can it run without a database or anything stateful? I'm using LiteLLM but it is not trivial to run.

[apatheticonion]

You can configure it with just config files.

Database is optional, if you add one, you get advanced caching.

It's just a single executable.

    wget https://downloads.getmaxim.ai/bifrost/v1.6.5/linux/amd64/bifrost-http
    chmod +x bifrost-http
    ./bifrost-http

[schneehertz]

openrouter’s API has removed parameters specific to certain models, impairing model functionality.

For example, the image generation parameters of GPT-image-2 are largely ineffective.

[ronsor]

Have you looked into OpenCode Zen or OpenCode Go?

https://opencode.ai/zen

[criley2]

Zen is nice, but they require US hosting so they don't get new Chinese models right away. There is no Kimi K3.

Go is nice for the ten minutes you can use it until your hit your cap.

[kzrdude]

Go is great if you prefer deepseek V4 flash. Then it's very hard to use up the allowance.

[samtp]

there is also https://cline.bot/cline-pass

[verdverm]

basically the same deal as OC Go, lending credence to token commodification

[adamisnotroman]

I didn't realize that OpenRouter had a mark up. Is it a flat mark up across the board or depending on the model?

[apatheticonion]

Under $20 it's a flat $0.8 fee. Over $20 it's +5.5%

+ credit card charges (+1.5%)

[bellowsgulch]

[flagged]

[dang]

"Don't be snarky."

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

https://news.ycombinator.com/newsguidelines.html

[apatheticonion]

It's more just the value for money. It doesn't really offer me anything valuable other than billing consolidation - and the markup is excessive for that use case.

If I was using the routing features it would make more sense - but they charge significantly more for that so the markup is really just for billing consolidation.

[Der_Einzige]

"After all, anyone can build an OpenAI API compatible server that works with every major provider, and even build it using an LLM at that!"

This but UNIRONICALLY lmao!!!