[rgaino]

I came here to ask about the cost of this based on the number of tokens. Since this prompt is repeated on every single request, isn't there a way to embed it when you load the model? Or they simply use the raw OpenAI API like any of us mortals?

[jonplackett]

A thousand tokens in the prompt from OpenAI only costs $0.015

But if you have a million searches, each having a 10 message back and forth, just for the prompt alone that's going to cost you $150,000

Surely MS pays a lot less than the OpenAI cost though.