|
|
|
|
|
by Wowfunhappy
56 days ago
|
|
Prompt caching is done on the provider side. If you send two requests to a provider in short succession and the beginning of your second request is the same as your first (for example, because your second request is the continuation of an ongoing chat), the repeated tokens are much less expensive the second time. Obviously, your tool does not provide this. But I think GP is undervaluing the UX advantages of having your conversation history. |
|