[shaky]

This is something that I think about quite a bit and am grateful for this write-up. The amount of friction to get privacy today is astounding.

[Aurornis]

> The amount of friction to get privacy today is astounding

I don't understand this.

It's easy to get a local LLM running with a couple commands in the terminal. There are multiple local LLM runners to choose from.

This blog post introduces some additional tools for sandboxed code execution and browser automation, but you don't need those to get started with local LLMs.

There are multiple local options. This one is easy to start with: https://ollama.com/

[apitman]

> It's easy to get a local LLM running

Easy for what percentage of people?

[sneak]

This writeup has nothing of the sort and is not helpful toward that goal.

[frank_nitti]

I'd assume they are referring to being able to run your own workloads in a home-built system, rather then surrendering that ownership to the tech giants alone

[Imustaskforhelp]

Also you get a sort of complete privacy that the data never leaves your home too whereas at best you would have to trust the AI cloud providers that they are not training or storing that data.

Its just more freedom and privacy in that matter.

[doctorpangloss]

The entire stack involved sends so much telemetry.

[frank_nitti]

This, in particular, is a big motivator and rewarding factor in getting local setup and working. Turning off the internet and seeing everything run end to end is a joy

[doctorpangloss]

NVIDIA drivers send detailed telemetry.

Windows and macOS send detailed telemetry.

You have to install the pip packages and the models, which all come from websites, which collect detailed telemetry.

You don’t think Microsoft gathers detailed telemetry on all your interactions with GitHub?

The local setup doesn’t really help with that.

[wkat4242]

> whereas at best you would have to trust the AI cloud providers that they are not training or storing that data.

Yeah, about that. They even illegally torrented entire databases, hide their crawlers. Crawl entire newspaper archives without permission. They didn't respect the rights of big media companies. But they're going to respect the little guy's of course because it says to in the T&Cs. Uh-huh.

Also, openai already admitted that they do store "deleted" content and temporary chats.

[Imustaskforhelp]

I agree but I was just (repeating?) some argument that I heard that if the companies would actually not follow on their premise that they are actually safe if they said so (think amazon bedrock tos policy which says such)

Then it will cause an insane backlash and nobody would use the product. So it is in their interest to not train/record.

But yes I also agree with you. They are already torrenting :/ So pretty sure if they can do illegal stuff scott free, they might do this too idk,

And yeah this was why I was actually saying that local matters more tbh. You just get rid of such headache.

[wkat4242]

> Then it will cause an insane backlash and nobody would use the product. So it is in their interest to not train/record.

I don't think there would be that much backlash. People are getting hooked on it and many don't actually care about privacy.

We know about Google, meta and people still use them. Not a big dent in openai usage either since their revelations.

But I understand your point!