Hacker News new | ask | show | jobs
by spullara 582 days ago
21MB? to split text? have you analyzed the footprint?
1 comments

Just to clarify, the 21MB is the size of the package itself! Other package sizes are way larger.

Memory footprint of the chunking itself would vary widely based on the dataset, and it's not something we tested on... usually other providers don't test it either, as long as it doesn't bust up the computer/server.

If saving memory during runtime is important for your application, let me know! I'd run some benchmarks for it...

Thanks!