[_boffin_]

From the initial look, it looks like you're taking the work of SweepAI, a source-available repo, which gave(?) it to LlamaIndex(1) and then you're just providing a wrapper around the LlamaIndex that provides a way to do an Update statement.

1) https://docs.sweep.dev/blogs/chunking-improvements

[ricklamers]

Sweep is mentioned as attribution in multiple places a) https://github.com/definitive-io/code-indexer-loop#attributi... b) https://github.com/definitive-io/code-indexer-loop/blob/fd9d...

The difference is packaging it as a consumable PyPI package that can easily be used in a project (they even call out for separating this out into a stand alone project but that they lack the time to do so: https://docs.sweep.dev/blogs/chunking-2m-files#future- )

In addition, we expand and fix the implementation, for example it now supports limiting on token count instead of character count, and we fix some white space inconsistencies in parsing/chunk reconstruction.

[_boffin_]

Looking forward to diving further in and seeing how you handle white spaces issues as I’ve had massive issues with that already

[ricklamers]

Please let me know if anything isn’t as expected and I’ll try to look into it