[muzani]

Could you just do this with LM Studio? It supports local servers too. I'm not sure about image gen but Llama 3 is tolerable with a MacBook Pro 2022 M1 with 32 GB RAM and no other GPU. Surely someone has an old MBP lying around for this.

But it's been 6 months since I used it and I feel like the proprietary ones (gpt-mini/o1-mini, claude-sonnet, gemini-flash) are just so much faster and cheaper than self-hosted. The real value is that your data remains private and the model doesn't silently change from 3.5 to 3.5-new.

What do you plan to use for "uncensored chat?" Many of the open source stuff are trained to be censored and I've had much better luck trying to get recent OpenAI chats to be uncensored.

[nancy_le]

i want a private AI for internal use to ensure it's private, instead of using OpenAI. There's something about business confidentiality and cost, so I came to the decision to create a local one