[numba888]

> Not sure if what I said makes sense.

Not sure either. But here is the lesson from this and other sources. To improve the output use multistep approach. Get the first answer, one or more, and pass it through the second verification step(s). Like 'for this * is this *' relevant? Or is it correct, does it solve the problem, etc.. Then select the answer with the best scores on the filters. You see, it's very similar to that in the original post. Get first candidates, filter.

[babyent]

Any way to speed it up? So of course I can try (keyword) to improve the results, but it all comes down to how well the embedding is stored, the quality of the actual user content, and of course my prompt.

I have a multi-step workflow already, but it is getting slow now going through all those steps. And sometimes, if the result is wildly incorrect, it feels really bad.

Again, I'm not an expert in this field but I am trying to learn and improve my product.

[numba888]

You can try using online services with best models. With prices like $1 for 10K requests it's hard to make local competitive solution. I'm going to use this approach in my personal projects.

[babyent]

Great! Same I’m using OpenAI models, not local. What about you?