[fingerthieff]

That's good to know, there are ways around the limit of course by breaking up the prompt into multiple messages and then you're at the mercy of the models context window which can be anywhere from 4k to millions.

At some point though like you say, it's going to become ineffective and you'd probably want to use the "Sampling" mode that is available to only send a random subset of your library to model as a general representation. Though how well this works on massive library remains to be seen.