[feanaro]

Many of your concerns could be alleviated by using Tree-Sitter. (https://tree-sitter.github.io/tree-sitter/)

[pfdietz]

Tree-sitter is great, but I find it could do a better job with broken code. This is particularly important when parsing things like C or C++ where the preprocessor makes it likely that unpreprocessed code can't be parsed anyway.