[wkat4242]

Yeah I'm specifically interested in this because I'm in a lot of local telegram groups which I have no patience to catch up on every day. I'd love to have ChatGPT summarise it for me based on a list of topics I care about.

Sadly the cost of GPT-4 (even turbo) tends to balloon for this usecase. And GPT-3.5-turbo while much cheaper and more than accurate enough, has a context window that's too shallow.

I wonder if Telegram will add this kind of feature also for premium users (which I also subscribe to) but I imagine it won't work at the current pricing levels. But it would be nice not having to build it myself.

[esperent]

GPT3.5 and GPT4 are not the only options though, right? I don't follow that closely but there must be other models with longer context length that are roughly GPT3.5 quality by now, and they even probably use the same API.

[wkat4242]

I don't really know. The benefit of ChatGPT is that it's so big, there are so many nice APIs for it :)

I'm not so deep into it all.

[ajcp]

Mistral 8x7b has can handle context of ~32,000 pretty comfortably and it benchmarks at or above GPT3.5

[ComputerGuru]

Is that the sliding context window size? Because I didn't have good results with sliding context windows in the regular Mistral models.

[ajcp]

Yeah, I think they fine-tune without a specific window size target to achieve and then keep expanding context until it starts falling over.