Hacker News new | ask | show | jobs
by estebank 26 days ago
Rust macros are Token Trees and provide namespace hygiene, so not quite "text in, text out".
2 comments

Token list, not token trees. There are official libraries for parsing token stream as rust code but you can parse it as anything (eg json, html) if you want to.
I think you meant TokenStream. They are trees, behind the scenes, because matching delimiters happens early on between lexing and parsing. By the time the rustc_proc_macro::TokenStream is exposed, the rustc_ast::tokenstream::TokenTree is hidden to the proc macro API.

https://doc.rust-lang.org/stable/nightly-rustc/rustc_ast/tok...

https://doc.rust-lang.org/stable/nightly-rustc/rustc_proc_ma...

You know, I was going to say tokens rather than text, but the AI discourse has me so burnt out on the term that I edited it. Regardless, one can emit unsafe blocks from a macro, provided they are valid tokens.