|
Perhaps not quite what you're asking for, but along the same lines there's this "Incomplete JSON" parser, which takes a string of JSON as it's coming out of an LLM and parses it into as much data as it can get. Useful for building streaming UI's, for instance it is used on https://rexipie.com quite extensively. https://gist.github.com/JacksonKearl/6778c02bf85495d1e39291c... Some example test cases: { input: '[{"a": 0, "b":', output: [{ a: 0 }] },
{ input: '[{"a": 0, "b": 1', output: [{ a: 0, b: 1 }] },
{ input: "[{},", output: [{}] },
{ input: "[{},1", output: [{}, 1] },
{ input: '[{},"', output: [{}, ""] },
{ input: '[{},"abc', output: [{}, "abc"] },
Work could be done to optimize it, for instance add streaming support. But the cycles consumed either way is minimal for LLM-output-length=constrained JSON.Fun fact: as best I can tell, GPT-4 is entirely unable to synthesize code to accomplish this task. Perhaps that will change as this implementation is made public, I do not know. |