Hacker News new | ask | show | jobs
by thomasfromcdnjs 752 days ago
Thanks for the answer.

Wikipedia has a lot of tables so I was wondering if content-aware sentence chunking would be good enough for Wikipedia.

https://www.pinecone.io/learn/chunking-strategies/

1 comments

mwparserfromhell can parse the text content without including tables