[berkes]

I googled* it and dod the same. But that merely moves the problem forward. Now the API returns errors that it's too large.

A smarter client would use a tool chain in which the first step is a model that's good at taking large contexts/data and extracting actual content from it. Many sites have a very low S/N ratio (readable content / dom).

Then pass that content, eg markdown, along to a model that's optimized at getting relevant parts out of content for the task at hand.

And only then onto the generic model to "do stuff" with it.

But many clients, including afaiks the Firefox one, just send the entire dom or html along to a generic model.

*Well DuckDuckGo

[prophesi]

Sorry for the late response; from what I can tell from my experience is that they're good at eliminating the unnecessary information (at least, as best as it can). You can see exactly what is passed in to the chat bot (for me, openweb-ui) and can see it's all text elements of what makes pertinent sense to be summarized.

My ideal would be an additional prompt that says "hey this page content is beyond your context length, fine with extending it to whatever the maximum context length plus some for the model's context length"?