[janpieterz]

How would I do this reliably? Eg give me 10 different values, all in one prompt for performance reasons?

Might not need JSON but whatever format it outputs, it needs to be reliable.

[jncfhnb]

Don’t do it all in one prompt.

[janpieterz]

Right, but now I’m basically running a huge performance hit, need to parallelize my queries etc.

I was parsing a document recently, 10-ish questions for 1 document, would make things expensive.

Might be what’s needed but not ideal.

[jncfhnb]

LLM performance is a function of the number of tokens, not queries