[avereveard]

at some point they will move from scanning files to scannign the AST and then token consumption will be greatly reduced by default, the challenge is that then you need something generic enough like tree-sitter to reduce the monumental effort of integrating a number of parsers.

[layer8]

Why would an AST greatly reduce LLM token consumption?

[avereveard]

a lot of token are used reading files whole just to understand where to fit the feature requested and the edit point, access to an AST would allow the llm to see the project "wireframe" so to say, by asking classes or method level granularity, and only then retrieving the source for the symbol that most likely contains the edit point the llm needs. some token consumption there is anovaidable as the llm need the source to build a diff, but still, it's focused on the feature than the search.

[bicepjai]

More focus on what to use rather when the whole file where the code snippet sits

[layer8]

I see, it could in principle more easily prune subtree that aren’t relevant. Initially I was assuming that the LLM would still ingest the whole AST in some form, since OP wrote “scanning the AST”. Does that mean the LLM would be invoking some sort of tool to perform a query on the AST?