[forgot]

Parser Generators have a bad reputation because they provide a leaky abstraction. In order to use an LR parser generator effectively you need to understand which languages the algorithm can handle.

Once you do, most of the problems vanish. For instance, in my experience, shift/reduce conflicts work almost like an integrated debugger. They highlight ambiguities in your grammar and most parser generators can produce a detailed report of the conflicts.

If, however, you do not understand the algorithm you are using and are for instance formulating your grammar to be completely right linear... well, you will get spurious shift/reduce conflicts.

The same applies to the lexer. Yes, a lexer can only process regular languages. Anything beyond that can be done in the parser. The parse tree generated by a tool like yacc is often messy and full of details. You can handle this by having an additional, hand written, pass which transforms the yacc output into the actual internal data structure of your compiler. Which, outside of toy examples, had better not be an abstract syntax tree. :)