[jakecodes]

Hey! It's awesome to read other people's solutions to this.

I've been working on solving this for the past 2 years or so and I went through much of the same struggles in the beginning until we came up with a solution which is fairly complex, to get LLM's to output data in a way we can use.

The big problem is that 95% accuracy is not good enough for calendars. People lose confidence after 1 failed attempt. Trying to get LLM's to output JSON can have a 1 in 1000 invalid JSON problem which is unrecoverable. What I wound up doing is training models for the tasks with tremendous amounts of data. I did not use OpenAI's models as they were not right for the job. Would love feedback.

convoke.ai

[ljlolel]

You can just force it to output into a specified BNF grammar this is quite easy

https://www.imaurer.com/llama-cpp-grammars/

[mofosyne]

Due to the vagueness of human language, could we also output a degree of confidence in the translation