[janpieterz]

Right, but now I’m basically running a huge performance hit, need to parallelize my queries etc.

I was parsing a document recently, 10-ish questions for 1 document, would make things expensive.

Might be what’s needed but not ideal.

[jncfhnb]

LLM performance is a function of the number of tokens, not queries