[NitpickLawyer]

Constrained generation guarantees syntax. It does not guarantee semantic correctness tho. Imagine you want a json object with "hp" and "damage". If you use a grammar, the model will be forced to output a json object with those two values. But it's not guaranteed to get sensible values.

With a 2nd pass you basically "condition" it on the text right above, hoping to get better semantic understanding.

[lyu07282]

I'm pretty sure the grammar is generated from the Json schema, it doesn't just constrain json syntax, it constraints on the schema (including enums and such). The schema is also given to the model (at least in openai) you can put instructions in the json schema as well that will be taken into account.

[NitpickLawyer]

Perhaps I worded that poorly. What I mean by semantic correctness is that the model could output nonsensical values for some things. Say in a game, "normal" health is ~100hp and the model creates a wizard with 50hp but then a mouse with 10000hp. So you're guaranteed to get a parsable json object (syntactically correct) but what the values are in that json is not guaranteed to make sense in the given context.

[viralpraxis]

You can specify `minimum` and `maximum` property for these fields. So this schema

  {
    "$id": "https://example.com/test.schema.json",
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "title": "Person",
    "type": "object",
    "properties": {
      "hp": {
        "type": "integer",
        "description": "HP",
        "minimum": 1,
        "maximum": 15
      }
    }
  }

is converted to this BNF-like representation:

  hp ::= ([1-9] | "1" [0-5]) space
  hp-kv ::= "\"hp\"" space ":" space hp
  root ::= "{" space  (hp-kv )? "}" space
  space ::= | " " | "\n"{1,2} [ \t]{0,20}

[michaelgiba]

For anyone curious here is an interactive write up about this http://michaelgiba.com/grammar-based/index.html

[lubujackson]

I find it does pretty well given a reasonable prompt and (especially) well-named keys/JSON structure. So if you had boss.mouse.hp you would get higher HP than random_enemies.mouse.hp, or better: enemies.level_1.mouse.hp.