[gopher_space]

This is software development, not sales. We rely on our tooling.

If I’m using a calculator to verify my math, I don’t want to use a second calculator to verify the first one.

[stale2002]

I am sorry to be the one to tell you but it was already the case that you cannot trust LLMs to solve all your problems 100% of the time.

It was always random. This is no different than any other randomness that already exists in LLMS.

If you are concerned just do benchmarks and see if it is valuable for your usecase regardless.

[thinkingtoilet]

Oh come on. All that happens is that it kicks the query to a model that was literally state of the art two days ago. Stop with dramatics.

[gopher_space]

We're hovering around the point that differentiates software developers from software engineers. If you create tools that people use to e.g. make or receive an income, moral and legal standards require this level of focus and commitment.

Because of this there's a chain of trust between myself and the tools I rely on to do work. The people who create those tools see unpredictability as a problem, and that's the only reason I'm using them. I can't work on important systems with a vendor product like Claude Fable.

That being said there's plenty of work to do where it'd be amazing. This isn't an either/or situation.