[nullc]

Using AI for making decisions create a whole host of novel vulnerabilities.

I've seen a lot of people talk about the obvious privacy and autonomy disaster it creates -- giving companies (and anyone who can legally compel or hack them) such intimate details or allowing yourself to be dependent on their services which they can withdraw or degrade at any time... Or where the model operators can bias the results based on bribes (ad payments) or their own agenda ... but really that's the start.

Another one what I haven't seen discussed is that use of an AI provides oracle access to your opponents. What I mean is that if I want to influence you, I can have an AI try variations of arguments against the model you're using (or if I don't know, a panel of models you might be using) until it can construct a super influential output. I can see what your model might recommend and anticipate your actions and change my actions to control yours. This can be done in a targeted way against individual people, or against populations e.g. by marketers.

There are reasons why this may be even more effective than it seems: It appears that models themselves are trained on each other, so in testing I see in family effects: GPT-4o mini seems much more convinced by essays written by GPT5 than ones written by GLM 5.2, while other models are more neutral (and my human review says the GPT5 essays are unambiguously worse in this trial). So I assume GPT5's writing was refined by review by in-family models and it's already somewhat of a super persuader against GPT-4o mini as a result.

The danger of oracle access is somewhat a symptom of AI monoculture resulting in a reasoning monoculture. If there were a billion highly distinct models widely used then it might be the only way to convince or predict a disproportionate portion of them would be to simply be right, and then we'd just be left with a operational security concern of making sure your opponents/competitors don't know what model(s) you use. But of course, that isn't the world we have today: people use a small number of models, and their behavior is generally pretty related due to architectural commonalities and cross distillation and such.