But isnt the information somehow cached when you start a new chat and build context with say GPT4? If the caching was so large as you say so many chat sessions in parallel would not be possible.
That's not my understanding. We can't be sure how OpenAI does things themselves, but adding messages to a conversation in the API means just rerunning the history through the prompt every time