[mhb]

You really need to abandon the "just keep it in the box" argument (i.e., "why give it such a level of control"). It's already out of the box.

Even if it weren't it could trivially convince or fool one of the hundreds/thousands of people working on it to do what it wants. In light of recent events, it is not reassuring to think that the people developing the next generations of AI have both the will and insight to even know where the box is exactly.

[peteradio]

A motivated individual could repeat the effort on city power, diesel supplement, a mishmash of cpus cobbled together to few thousands of nodes and a cleverly selected training set... And then connect it to the Internet on purpose. Frankly I think the eventuality of this are llm created firewalls of viruses consuming ever increasing amounts of energy to destroy any uninitiated/unpaying connector. Across the world knots will form at the only borders that matter anymore. Eventually one ID provider will protect us all from noone and the only people who need to worry are those who incidentally wind up on the wrong side of everywhere.