[threeseed]

You are conflating asking a single question to ChatGPT versus AI agents which typically need to interact with an LLM multiple times.

And the 5-10% is on average and gets significantly worse as you expand the context length which is also something you want for an agent.

[mewpmewp2]

It depends on the problem right. It would have 0 accuracy one some problems and near 100 percent on others.

Based on what you are attempting to do you could get any average in the end.