There is nothing totalitarian about constraining societal harm.
The issue comes down to whether it is collectively understood to be a benefit to the human race. Until now we have never had to constrain information to protect ourselves.
Please read the Vulnerable World Hypothesis by Nick Bostrom
> There is nothing totalitarian about constraining societal harm.
Of course not. The problem is that the only way to enforce AI regulations is through totalitarian means.
You can easily regulate OpenAI and Gemini and whatnot, but then people will just use local AI models. The barrier to entry for using local AI models is basically zero because software like Ollama make it trivially easy to set up, and small LLMs can run on an iphone or a laptop.
The only way to actually prevent "unauthorized" AI usage is to control digital communications to prevent LLM weights from being distributed and to ensure that no "black market" AIs are being trained or used.
And if you're already scanning digital communications for LLM weights, why not also scan it for other forms of wrongthink?. Hey presto, now you have totalitarianism.
I don't think that LLMs fall into the category of an infohazard in the way that Bostrom defines it. It presents a risk, but not one severe enough to justify universal surveillance. Bostrom is talking about AI that can make bioweapons, not ones that gives false confidence to incompetent people.
The issue comes down to whether it is collectively understood to be a benefit to the human race. Until now we have never had to constrain information to protect ourselves.
Please read the Vulnerable World Hypothesis by Nick Bostrom