The idea of asking ChatGPT and sharing my financial or medical data with OpenAI, or asking Gemini and sharing it with Google, or any other cloud AI provider doesn’t sit well with me. How about you?
I tried Ollama but found requests a bit slow since it doesn't seem to fully utilize my resources. LM Studio has been better for me. It uses the upstream GGML implementation, which feels more optimized. My problem with LM Studio is that they've added so many options and features lately that it takes a lot of configuration.
What's your favorite open model for writing and coding these days?
I'm using phi4 for writing, one of the qwen3s for coding and a small mistral for classification and other small tasks. Have a framework desktop showing up soon, will put a 70b/80b multimodal on it for image and pdf processing.
I have used ollama, lmstudio, jan and vllm at different times, am readying for a wholesale transition to llamacpp.
I run Qwen3 locally for coding and writing. It’s a solid model.
Framework Desktop is becoming really popular. The one with the Max+ 395 and 128GB of RAM is an absolute beast. I might buy a Beelink GTR9 Pro (Max+ 395 with 128GB RAM), which costs around $2,000.
llama.cpp is the real deal. I’m using it as the engine for the product I’m building right now (https://tygra.ai/).
Not comfortable at all, but when you have serious medical issues this is the least of your problems.
For work, I assume everything is used for corporate espionage, so it depends on the sensitivity of the data. If my employers / clients authorize a tool it becomes their problem.
No, I'm not talking about serious medical issues. I mean uploading your blood test results and asking an AI, "Give me specific nutrition, lifestyle, and supplementation recommendations."
For work, as an employee, sure, it's easy to say the company approved ChatGPT or Gemini, so you can go ahead and upload, for example, usage data to get a retention analysis. But what if you're the employer?
I don't worry about it. The incremental loss of privacy is approximately zero, as far as I'm concerned.
I assume I have no privacy these days. I've got Amazon sponsored spies in my house, listening at all times. I've got a completely insecure computer/operating system and an internet provider known to sniff traffic to sell info.
I've got a backup of everything on Backblaze's servers.. I assume the NSA has a copy of everything I've ever done... I joke they should just offer free backup service to everyone, and save some duplication. ;)
I don't have any secrets, I can't in the modern world.
This means I don't really have to worry, I'm free to just go about life online.
The first thing to keep in mind is the illusion of transparency. You might internally know that something is wrong or exploitable or you've made an obvious mistake, but that's generally much less obvious to others.
The second to keep in mind is that we are currently in a crisis of attention. There's too much to think about and do nowadays, and there is a gigantic lack of motivated actors to act upon that information. You could consider it the dual of the illusion of transparency, but it's the illusion of motivation. Other people, by in large, just do not give a damn because they can't and don't have time for it.
Even a nation state if they wanted to go spy on everyone's private information would immediately find themselves with too much nonsense to sift through and not enough time to actually follow through even on surface level information. Let alone leaks that actually require some sort of sophisticated synthesis over two or three disparate pieces of info.
Lastly, it's the difficulty in exploitation. You know how projects and code and stuff seem easy until you try them, and it turns out that actually, this is taking forever, and it barely works? The whole devil in the details thing.
Well, that applies to exploits as well. It's easy until you try it, and then you have this Swiss cheese model of success where random stuff doesn't line up correctly and your workflow broke.
AI surveillance btw barely changes any of this calculus.
If you can run an open-source LLM locally on your own computer, completely offline, and use it for legal, finance, or medical topics, would you still say no to that?
I can and have when I was required to. It was slow and had worse results than I had hoped. Probably because I dont have enough VRAM for the big open source models so I was using 8b ones.
I left this bit out because my original comment was getting long but I think it's important to be respectful over others' privacy wishes. So I didn't use an API when it concerns other people.
Of course, you need decent hardware to run LLMs locally, but you don’t need a super high-end computer to host qwen3:30b or gpt-oss:20b. You don’t even need a GPU for those models, as long as you’ve got a modern CPU. And they’re already pretty solid for writing and coding.
No, but I've done it by mistake. I wanted chatgpt to proofread a letter and uploaded it without think over the consequences. It's very possible to do it if you aren't careful. Keep that in mind.
Sam Altman actually said this on a podcast with Theo Von recently.
“So if you go talk to ChatGPT about your most sensitive stuff and then there’s like a lawsuit or whatever, we could be required to produce that,” Sam told Theo.
He even asked Theo about his own ChatGPT usage, and Theo admitted he doesn’t use it much because of privacy concerns.
Sam’s response:
“I think it makes sense... to really want the privacy clarity before you use [ChatGPT] a lot — like the legal clarity.”
10 years ago I was reluctant to use smartphone browser, because it's obvious everything is tracked and profiled as on mobile there was no ad blocker, no possibility to edit hosts file. Now I just use mobile browser for everything without second thought. Give it 3-5 years.
I don't think we're comparing apples to apples here. Using a mobile browser certainly opened the door to more tracking, but at the end of the day it was mostly about ads and profiling. With AI, the scale is very different. These systems can learn far more about you from the questions you ask and the data you provide. That makes privacy and data protection a much bigger concern than ever before.
But I'm not sure that in 3-5 years people will just give their sensitive financial, medical, or legal data to cloud AI providers without a second thought.
I usually try to anonymize whatever is going on. Personal conversations key details removed or slightly modified, no real person or company names at all, etc
Code I only run through the zero-retention API accounts anyway.
Just last week I wanted to do some financial analysis with ChatGPT (GPT-5), but I wasn't comfortable uploading the CSV with all my transaction data. Instead, I used Qwen3 running locally on my MacBook to anonymize the data first, and then uploaded the sanitized version to ChatGPT.
Nope, I have done it though. The UK govt is making us all upload identity documents to every website so I figured all these sites may as well have all my information.
"All these sites may as well have all my information." I don't think so. That's scary!
But you do have a choice.
If you want to upload your bank statements and ask an AI, "How did my spending habits change in Q2 compared to Q1?" or upload your blood test results and ask, “Give me specific nutrition, lifestyle, and supplementation recommendations," you can run LLMs locally. No data will leave your computer.
Work: Because your company is using Microsoft 365, you can upload customer data to ChatGPT - for example, for revenue reporting. Am I understanding you correctly?
Personal: So, you download and run open-source LLMs on your computer?
We have confidentiality agreements with Microsoft that are worth whatever lawyers think they are worth. Using chatGPT though the company azure accounts is fine, the entreprise Microsoft copilot is fine too, but standard ChatGPT isn’t fine.
Personally no I dont use LLMs for confidential data, the local ones that can run on my personal computers aren’t good enough.
Give qwen3:30b and gpt-oss:20b a try. You don't need a fancy GPU, just a modern CPU. Those models are already pretty solid for analyzing your personal data.
Not a wrapper of ChatGPT. I mean no cloud AI at all for sensitive data. A different approach: local AI. A desktop app where you can download the open models you want and run them locally. 100% private, no data ever leaves your computer or network. The real question is whether companies would pay for team features on top of that.
Investing quite a lot of time to figure out hosting LLMs locally in comparable quality without investing too much money (as smaller environments will never have a large budget for this).
Obviously you need decent hardware to run LLMs locally, but you don’t need a super high-end computer just to host qwen3:30b or gpt-oss:20b. Those models are already pretty solid for writing and coding.
Having read the myriad AWS data protection agreements I would feel comfortable running bedrock hosted models. Others may feel differently.