[ptx]

> it eliminates entire swaths of syntactical errors, allowing the LLM to focus on the semantics

As I understood it, the idea the article presents is basically the opposite: The LLM is completely clueless about the semantics, but by constraining the language so that semantic errors are syntax errors, we can catch the LLM's bullshit with a simple validation step and make it try again until it produces valid syntax, which limits how wrong it can get the semantics.

It does apply to JSON in a way: An LLM doesn't always produce valid JSON, but it's easy to do a syntax check with a JSON parser and make the LLM try again until the JSON is valid. The JSON might still be semantically wrong on the level of the application domain (have the wrong values), but at least you can be sure that the generated data consists of lists, dictionaries, strings and numbers.

[tingletech]

I thought they got like a logit mask that removes illegal next tokens, at least in llama.cpp and GBNF.

[ptx]

You're probably right about that. It looks like llama.cpp lets you specify a grammar file or a JSON schema - is that what you mean? If the syntax of your DSL can be expressed that way, I suppose you wouldn't have to validate it externally and retry.

[tingletech]

the grammar support/json-schema applies to a whole response, but I was under the impression a similar logit guide rail can be applied when a tool call is opened.