[babyent]

Any way to speed it up? So of course I can try (keyword) to improve the results, but it all comes down to how well the embedding is stored, the quality of the actual user content, and of course my prompt.

I have a multi-step workflow already, but it is getting slow now going through all those steps. And sometimes, if the result is wildly incorrect, it feels really bad.

Again, I'm not an expert in this field but I am trying to learn and improve my product.

[numba888]

You can try using online services with best models. With prices like $1 for 10K requests it's hard to make local competitive solution. I'm going to use this approach in my personal projects.

[babyent]

Great! Same I’m using OpenAI models, not local. What about you?