[xrd]

I want this for my kids when I'm driving them to school. We try to play dungeons and dragons and I'm exhausted trying to drive and be DM. Is there an open source LLM (Llama?) which could be configured to do this? I hate operating against a commercial service.

[spudlyo]

There are a number of models based on Meta's Llama that have been fine-tuned for storytelling. These are not truly open source per-se, due to the murky nature of the leaked base model, but should be fine for what you're describing.

If you're interested in this sort of thing I'd recommend reading r/LocalLLaMA on Reddit and checking out @samwitteveenai on YouTube. Local inference of the larger 30B models is going to require either a fairly beefy Apple M1/M2 machine or a PC with lots of memory and one or more 3090/4090 type GPUs. The smaller models are improving all the time though, as advancements in fine-tuning seem to be happening very quickly.

You can learn a lot using a Google Colab notebook (you can get one with a A100 GPU for less than you'd expect) and playing around with the langchain tools. If your D&D game has a reasonable amount of written lore, it might be a lot of fun to use this as an opportunity to learn about LoRa training.

[xrd]

Thanks, this is great information. Really appreciate the thorough answer.