[wrs]

I guess I don't really understand the goal of "one-shot" parser generation, since I can't even do that as a human using a parser generator! There's always an iterative process, as I find out how the language I wanted isn't quite the language I defined. Having somebody or something else write tests actually helps with that problem, as it'll exercise grammar cases outside my mental happy path.

[Verdex]

The comment that started this whole thread off mentioned LLMs oneshot-ing parsers. I didn't think an LLM could one shot a parser and I am interested in parsers which is why I asked about more info.

It's not a goal of mine but because of interests in parsing I wanted to know if this was something that was happening or if it was hyperbole.

[hombre_fatal]

By one-shot in my original post, I mean that it comes up with a decent, working implementation that I can then refine.

As opposed to getting there through incremental revisions where I must dictate the directions the LLM takes -- I don't necessary know the directions it should take because I might be agnostic about it.

I find success in breaking the problem down into tokenize and parse just like I do when writing parsers myself.

    tokenize(string) -> Token[]
    parse(Token[]) -> Node[]

And definitely get it to generate tests every step of the way. The insane part is when the LLM can run tests itself and can iterate on code until all tests pass.

Or you extend it with a feature just by writing some failing tests and it does the rest.

[wrs]

Well, I mean, it sort of did one-shot the parser in my case (with a few bugs, of course). It just didn't one-shot the parser I wanted, largely because my definition was unclear. It would be interesting to see how it did if I went to the trouble of giving it a truly rigorous prompt.