[shilangyu]

Are there any benefits for users that tree-sitter is used under the hood? Can we benefit from the killer features of tree-sitter? Namely incremental parsing, fallible parsing, lossless syntax tree, or being embeddable into editors supporting tree-sitter syntax highlighting?

[shadaj]

Yes! Right now, the main benefits are the ability to write grammar definitions that are quite close to the ideal AST structure (made possible by Tree Sitter's grammar format), and being able to embed the parser in many different applications (including WASM via https://github.com/shadaj/tree-sitter-c2rust). Rust Sitter also gives quite nice error diagnostics with spans thanks to Tree Sitter's recovery logic.

Fallible parsing is something I plan to implement in the very near future, by letting users wrap types in `Result` to mark them as an error boundary. Incremental parsing is a bit more difficult, since we'll need to add logic to know when an existing AST struct can be reused, but is on the roadmap.

[shilangyu]

I would like to delve into the compatibility with tree-sitter, since in other features tree-sitter being under the hood is mostly an implementation detail:

If I were to write my parser using rust-sitter, would I be able to still generate the final standalone tree-sitter parser as a `.so`? That way I could integrate with tools supporting tree-sitter parsers (for instance https://github.com/nvim-treesitter/nvim-treesitter#language-...) without having to write the `.js` grammar?

[shadaj]

In principle, yes, you can use the `rust-sitter-tool` crate to generate the Tree Sitter JSON definition and then compile it to a standalone parser. The grammar is auto-generated though so it may be a bit trickier to integrate into other tooling? The general problem of exporting just the grammar is something that's been on my radar, but haven't had a chance to think through it too deeply yet.