Hacker News new | ask | show | jobs
by aabdi 18 days ago
Fair, but I don’t see what case you have w this. Mind sharing?

Seems niche to be both uncacheable and long context?

1 comments

Anything where you're dealing with a large volume of records/documents. Lots of people are using these for large-scale digitization of documents - scanned stuff being OCR'ed and summarized, generating embeddings, etc. Large scale translation.

Anywhere where you might have a large backlog of data to work with can end up in this sort of situation.