| Honor to meet you! [Anonymous] founder of a similarly high-profile initiative here. > Creating a prompt to write JSON is possible but need quite an elaborate prompt and even then the LLM can make errors. We want to make sure that all JSON coming out of the model is directly actionable without having to ask the LLM what they might have meant for a specific value The LLM cannot make errors. The LLM spits out probabilities for the next tokens. What you do with it is up to you. You can make errors in how you handle this. Standard usages pick the most likely token, or a random token from the top many choices. You don't need to do that. You can pick ONLY words which are valid JSON, or even ONLY words which are JSON matching your favorite JSON format. This is a library which does this: https://github.com/outlines-dev/outlines The one piece of advice I will give: Do NOT neuter the AI like OpenAI did. There is a near-obsession to define "AI safety" as "not hurting my feelings" (as opposed to "not hacking my computer," "not launching nuclear missiles," or "not exterminating humanity."). For technical reasons, that makes them work much worse. For practical reasons, I like AIs with humanity and personality (much as the OP has). If it says something offensive, I won't break. AI safety, in this context, means validating that it's not: * setting my thermostat to 300 degrees centigrade * power-cycling my devices 100 times per second to break them * waking me in the middle of the night ... and similar. Also: * Big win if it fits on a single 16GB card, and especially not just NVidia. The cheapest way to run an LLM is an Intel Arc A770 16GB. The second-cheapest is an NVidia 4060 Ti 16GB * Azure gives a safer (not safe) way of running cloud-based models for people without that. I'm pretty sure there's a business model running these models safely too. |