[armchairhacker]

This is where open-source low-cost models shine and could be useful today.

These models aren’t very smart, but seem to be good at role-playing. Open Assistant actually has more personality than GPT (mostly because it can be a bit creative and unpredictable), plus says “as an AI language model” much less often.

More importantly, the ability to run them locally and/or cheaply means you can put them in ML side projects and games and unrelated apps. And since the code is open-source, advanced developers could even augment them to improve the memory and add plugins.

The main problems are that I believe most of these models don’t have a “system” prompt, and are also a lot worse at remembering things. But I’m sure the former is being worked on and explicitly encoding state (e.g. by reminding the chatbot in each prompt) can fix the latter.