[villedepommes]

> Go actually has Yacc as a built in tool (although know Lex).

Interesting. Did they combine both a lexer and a parser into one tool?

lex,flex, and re2c are typically standalone tools (lexers) that tokenize input, whereas yacc and bison are parsers that parse those tokens into ASTs (usually).

[throwaway2016a]

> Did they combine both a lexer and a parser into one tool?

No. They give you an interface definition for the lexer and you can implement the methods however you see fit as long as it matches the interface.

Which I found annoying as there are some tedious parts to writing a lexer as soon as you have a language requiring multi-rune tokens with common prefixes.[1]

Apparently the back story is that go needed a parser generator but they didn't need a lexer so they only built just the one out of necessity.

I read some places that they would welcome pull requests for a lex/flex equivalent.

[1] A `rune` in GoLang is like a `char` in C only it stores multi-byte characters where as in C it is more analogous to a single `byte`

[hnlmorg]

For clarity (I know you know what a rune is but your description felt a little misleading imo):

A rune is just an alias for int32. But it's context usually means unicode chars which are indeed multi-byte.

You can still use byte arrays (or "slice" if you're writing idiomatic Go, but that's another tangent again) too though.

Also "byte" in Go is another alias, but for int8. But that shouldn't come as a surprise to anyone :)

[throwaway2016a]

Thank you for the clarification. Your explanation will be helpful to people. I wasn't planning on going into that technical detail, I was focused more on typical usage.