[rahimnathwani]

Why would you want chunks that big for vector search? Wouldn't there be too much information in each chunk, making it harder to match a query to a concept within the chunk?

[nostrebored]

The problem is that often semantic meaning depends on state multiple paragraphs or sections away.

This is a coarse way to tackle that