Hacker News new | ask | show | jobs
by brennvin 1440 days ago
I have never understood why (some) people want to write lexerless parsers. Lexing lets you approach two different concerns separately. What is not to like?
4 comments

For haskell, I think a lot of it is just that parser combinators are _very_ satisfying and memorable. They're what made the whole Functor/Applicative/Monad thing really click for a lot of people as an interesting/useful thing beyond just IO.
Two reasons in the wild:

1. You might not know a priori that the lexical analysis can be done without any knowledge of the surrounding grammatical context.

2. You may be able to give more informative error messages if you know the grammatical context.

Agreed, and I would state it more strongly: it's unclear how / significant extra work to provide good error messages if you have a separate lexer stage. And sooner or later you will want good error messages.
For simple languages, it can be a bit of a pain to write and maintain a lexer and a parser. For example, if I'm gonna parse JSON and nothing more, I think it's a bit silly to use a two external DSLs when I could use a simple eDSL. I think Happy and Alex both have their places, absolutely. For parsing full on programming languages I can imagine using parser combinators to be a bit painful. But parser combinators work great for a lot of cases.
You can still have a lexing step when using a parsing combinator approach, and in practice you probably should for complex parsing tasks. Unfortunately, I'm not sure I've ever read a tutorial which really walks through doing such a thing.