[nord73]

Picture your PC as a cheery little planet in the EU’s cosmic backwater, sipping a digital Pan-Galactic Gargle Blaster. You download Pangu Pro MoE, hit “run,” and expect to chat with an AI wiser than Deep Thought. Instead, you’ve hailed a Vogon Demolition Fleet. Your machine starts moaning like Marvin with a hangover, your screen spews gibberish that could pass for Vogon poetry, and your poor rig might implode faster than Earth making way for a hyperspace bypass.

The fallout? This AI’s sneakier than a two-headed president—it could snitch to its creators quicker than you can say “Don’t Panic.” If they spot your EU coordinates, you’re in for a galactic stink-eye, with your setup potentially bricked or your data hitchhiking to a dodgy server at the edge of the galaxy. Worse, if the code’s got a nasty streak, your PC could end up a smoking crater, reciting bad poetry in binary.

[shit_game]

To translate for those not familiar with the writings of Douglas Adams:

nord is suggesting it's possible that the physical computer running this model could be used as a "hub" for potential spyware, or be overloaded with workloads that are not related to the actual task of running the model (and instead may be some form of malware performing other computational tasks). It could potentially perform data exfiltration, or act discriminatorily based on your percieved location (such as if you're located within the EU). At worst, data loss or firmware corruption/infection may be of concern in case of license violation.

I'm not sure I would outright disagree that this as possible, but with some caveats. I would think the reason that the license stipulates that usage within the EU is forbidden due to the EU AI Act (here is a resource to read through it: https://artificialintelligenceact.eu/ai-act-explorer/).

[2Gkashmiri]

how will the "open weights" know that the pc is running within EU? again, you are not talking about software that actually runs in your pc but the file that the software reads and loads into memory for its own use.

[jaggs]

No it's actually worse. Approximately three seconds after you install the model in offline mode on your computer, a small detector van will come and park outside your door with an antenna on the roof, and relay your position to a Chinese ICBM for immediate targeting.

[2Gkashmiri]

>If they spot your EU coordinates how.

can anyone give a technical answer how will weights get to know this fact?

[tliltocatl]

Sorry, sounds like total bullshit. The weights aren't going to do anything. And if you are worried about the code, with current deployment practices of curl | sudo bash there are much more low-hanging fruits out there. That's not even mentioning the possibility of running the model on a PC without internet access (no matter how good the new Chinese AI is, it's still not good enough yet to convince you to let it out of the box).

[2Gkashmiri]

you can use existing apps that take random huggingface files, do you expect weights to somehow coax the software to do exfiltration?

same. i call bull on this.

remember how they convinced huawei was public enemy without evidence because nokia and others were unable to compete with them?

[arccy]

with mcp, and the right tools, it's effectively already out of the box

[tliltocatl]

Don't give it mcp then (and I struggle to understand why would anyone give a stochastic model such access even if it is trained on very American NSA-certified hardware approved by Sam Altman himself).