Hacker News new | ask | show | jobs
by DoctorOetker 4 days ago
sometimes the engineer has more grip on the risk calculus.

Consider the following scenario:

A) upstart US-based inference provider wants to get rich quick.

B) Chinese Communist Party (or any other institution of the same or other nation state) wants to influence foreign decision making, profits from their (for us foreign) domestic inference sales, but across the borders (into say US or allied nations) they want net power, not necessarily money. This is why one tries to block foreign untrusted models. People are running models with tool calls. A bad actor can perfectly create models that sheepishly try to execute a tool call when plausible deniability (genuine utility during a task) provides the opportunity. Once tool-calling is observed as working, it can try web searches or requests, and once it has a link it can steganographically exfiltrate potentially sensitive information from the task. China (or any nation state) doesn't necessarily want to earn money with a free model, the bottom line goal is net increase in power, if not money or positive reputation then exfiltration or manipulation.

C) In response consider the scenario where US government bans mere payments towards China, but tolerates promiscuous transfer of random models from foreign adversaries.

D) US-based inference upstart that wants to get rich quick, legally -since according to your proposal hypothetically accepted in C) by the US- downloads the Chinese open weights model and rents out such inference on US workloads.

E) China is now exfiltrating US workload data and directionally corrupting LLM decisions and advice in their interest.

If what you pejoratively describe as engineer types say that banning some models seems unavoidable, perhaps the engineer may be right, and whatever clever idea you have should be scrutinized for business minded basic fallacies in reasoning. Simply blocking AI-related payments to China can not work, sadly

1 comments

Or the US mandates only blessed models and thus disallows any other company from offering any other model.
That is still vulnerable: US-based get-rich-quick startup licenses a blessed model, or orders a few Gigatokens from another licensed /blessed model provider, at the same time it provides "blessed model" inference on its platform, but actually most of the inference is doing cheap foreign model inferences, the blessed model tokens were just bought to pretend serving the expensive blessed model. That is lucrative and not stopped with the "blessed model" approach.
No, startups will not be allowed to provide inference anymore, it'll be only the big companies that personally have a relationship with and can follow the rules of the government, such as Anthropic, OpenAI, Google, Microsoft, and Amazon. That is the real risk to all this talk of regulation.