[akavi]

Typescript's type system is much more expressive than the one the function call feature makes available.

I imagine closing the loop (using the TS compiler to restrict token output weights) is in the works, though it's probably not totally trivial. You'd need:

* An incremental TS compiler that could report "valid" or "valid prefix" (ie, valid as long as the next token is not EOF)

* The ability to backtrack the model

Idk how hard either one piece is.

[rezonant]

For the TS compiler: If you took each generation step, closed any partial JSON objects (ie close any open `{`), checked that it was valid JSON and then validated it using a deep version of Partial<T>, that should do the trick.

[akavi]

Not for even the simplest schemas.

Eg, given even the type:

    {"aLongerKey": "value"}

The generation prefix:

    {"a

would by your algorithm produce the following invalid output:

    {"a}

[rezonant]

That's why I mentioned you check the JSON validity first. You'd obviously need to continue letting it generate tokens until you can parse the JSON to check if the type is partial. You could of course close even the quotes but then you'd get "not valid" signals from TS when the AI is like "just let me finish!" :-)

[just-ok]

But that isn’t valid JSON

[akavi]

Right, it would fail even before hitting the typing check.