[minimaxir]

You can do much bigger chunks with models that support RoPE embeddings, such as nomic-embed-text-1.5 which has a 8192 context length: https://huggingface.co/nomic-ai/nomic-embed-text-v1.5

In theory this would be an efficiency boost but the performance math can be tricky.

[qudat]

As far as I understand it, context length degrades llm performance, so just because an llm "supports" a large context length it basically just clips a top and bottom chunk and skips over the middle bits.

[rahimnathwani]

Why would you want chunks that big for vector search? Wouldn't there be too much information in each chunk, making it harder to match a query to a concept within the chunk?

[nostrebored]

The problem is that often semantic meaning depends on state multiple paragraphs or sections away.

This is a coarse way to tackle that