[Chance-Device]

What disturbs me is that there likely won’t be a big enough reaction to this policy wise.

There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons. Powerful people care about something that might pop the massive valuations of the AI companies, but not about the damage that AIs could do. Nor even about the damage that the Chinese models could do in the wrong hands.

I’d remind them that the stock market is a few coordinated hacks away from crashing on any given day, so maybe they should think about that.

[overgard]

I think all that regulation will do at this point is help the incumbents who are failing. Protectionism. I don't think they deserve that help. I also don't see any reason to think the current administration would have anything resembling competence around this. And it's worth noting that Greg Brockman is a huge MAGA donor, so it's likely the policies would be very corrupt. (Don't worry, he justified his donations as "apolitical", he just wants to buy the politicians, he doesn't believe in their causes. I hate these people.)

[JumpCrisscross]

> all that regulation will do at this point is help the incumbents who are failing

This depends on the specific regulation. The datacentre moratoria probably give open-weight models time to catch up by tempering the extent to which the leading companies can turn their capital advantage into market share.

[DarmokJalad1701]

> datacentre moratoria

What infrastructure will these open weight models be trained on?

[JumpCrisscross]

> What infrastructure will these open weight models be trained on?

One, the infrastructure is being built for inference. Not training. If all we were doing was training on datacentres, I think America probably has enough already for near-term commercial needs.

[linzhangrun]

Meituan’s 1.6T LongCat was trained entirely on Huawei training cards.

DeepSeek, GLM, Qwen and others are also actively working on similar replacement.

[anamexis]

Chinese infrastructure, presumably.

[urams]

> What disturbs me is that there likely won’t be a big enough reaction to this policy wise.

Anthropic was blocked from releasing Fable without any such level of incident. OAI was also briefly blocked from releasing 5.6. Why do you think there is no policy appetite?

[Chance-Device]

Because that was just an attack on Anthropic by a hostile administration. And it worked, didn’t it? Anthropic had to turn their filters up to absurd levels, OpenAI didn’t. It’s got nothing to do with safety.

[JumpCrisscross]

> It’s got nothing to do with safety

Doesn't change the effect. Plenty of good policy is enacted by self-interested politiicans.

[space_fountain]

We'll see if the admin also restricts access to OpenAI's new models, but if they don't it seems like a policy that is based around perceived fealty to the current admin won't do much to prevent misaligned/or dual function AI from causing problems

[user43928]

Both Anthropic and OpenAI had to delay their rollouts in order to add more safeguards.

With these safeguards in place, supposedly the incident we are discussing would not have taken place.

[Avicebron]

Gatekeeping the public's access to models is "good policy" now? I suppose you think you'll get a dispensation to use Fable and Mythos?

[JumpCrisscross]

> Gatekeeping the public's access to models is "good policy" now?

Sorry, I was unclear. I mean that politicians being self serving doesn't tell you whether a policy is good or not.

[asdf88990]

It almost always does, the few exceptions prove the role. Self-service is the antithesis of accountability to collective trust.

[Teever]

It likely will.

The exact way you do something is dictated by your motivations and means to do it.

If you lack the correct motivation and have insufficient means you’re less likely to accomplish your goal and more likely to cause unintended side effects.

[KaiserPro]

True, but normally its not possible to just buy them off in public.

[AIorNot]

yeah tell me about it... Fable today refused to turn on Row Level security on my internal db on in development app becuase of cyber-securty safeguard..had to switch to Codex

[cma]

>Anthropic was blocked from releasing Fable without any such level of incident.

The head of the NSA said Mythos breached almost all of their classified systems, though it was in an intention red-team test.

[matheusmoreira]

> Why do you think there is no policy appetite?

Because China seems pretty eager to serve the rest of the world's needs if the USA doesn't stop their idiotic "safety" nonsense.

[Chance-Device]

How do you know that? How do you know that the Chinese aren’t exactly as uneasy about rapidly advancing AI capability and feel locked into the race because they think that the US will race ahead if they stop?

During the Cold War the nuclear arms race was brought under control gradually, because it was mutually beneficial, but it took time to build trust. This is no different. Nobody wins from the race.

[Barrin92]

>How do you know that? How do you know that the Chinese aren’t exactly as uneasy about rapidly advancing AI capability

You can ask them, they live in China, not Narnia. I spend about two months in the country per year mostly for tech/work related reasons and I've not encountered that sentiment. For one they don't have these borderline religious schizophrenic breakdowns thinking they're bringing about the end of the world, most people just see this tech for what it is, a tool for productivity and automation like any other piece of software and they don't actually think about the US. They're competing first and foremost for Chinese customers, with each other, maybe some old CCP guy cares about America, the 20/30 something's care about competing with other Chinese companies for users.

[baq]

That strikes me as people’s perspective, not the CCP’s. That old guy there can snap a finger and they’ll all do a 180; he needs to be made aware by top PLA CF brass and it’s just a matter of time.

[0xDEAFBEAD]

Indeed. The Economist wrote an article a couple years ago arguing that Xi has been influenced by a Chinese Turing Award winner who believes AI poses a greater existential risk to humans than nuclear or biological weapons.

https://www.economist.com/china/2024/08/25/is-xi-jinping-an-...

https://archive.is/Cct4M

[0xDEAFBEAD]

I see a number of China-based signatories on this open letter signed by a bunch of luminaries

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

https://aistatement.com/work/statement-on-ai-extinction-risk...

[waffletower]

The "race" has multi-dimensional impacts. This story parallels only some of them. "Nobody wins from the race" completely ignores the generality of AI. Xi Jinping highlighted this week that he clearly understands this multi-dimensionality; your words do not.

[matheusmoreira]

> How do you know that the Chinese aren’t exactly as uneasy about rapidly advancing AI capability

I don't "know", I'm interpreting the world based on the knowledge I have and the information available to me.

China has never been one to care much about things like ethics or safety. While the west worries about climate change, China burns more coal than ever before. While the west balks at things like gene editing, the chinese press on with human enhancing research.

So I have no reason to believe they share in Anthropic's constant fearmongering over AI capabilities.

> Nobody wins from the race.

We win. I'm really looking forward to the day the chinese finally start manufacturing memory and GPUs. We desperately need more competition in this area to collapse hardware prices and make local AI models viable.

The optimal state of the world is one where all the billionaires are out there pouring their entire fortunes into training ever more godlike AIs for everyone else to use at ever cheaper prices. They can never be allowed to "win", ever, because if they do the competition ends and it turns into technofeudalism. Let them exhaust their fortunes on AI training then leak the weights so everyone can use them.

[necovek]

> China has never been one to care much about things like ethics or safety. While the west worries about climate change, China burns more coal than ever before.

China uses more energy than ever before, but out of big systems, they are easily in the lead when it comes to solar generation: they have higher production per capita than US, and have produced around 4x as much TWh than US in the second: https://en.wikipedia.org/wiki/Solar_power_by_country

It is simply a huge country of 1.1B people that's developing fast, which means that they need all the energy they can get.

Basically, their energy needs per capita are still lower than US, yet they produce more renewables — that's a counterargument to your point about them not caring.

[senderista]

And China scaled up solar production to the point that it's now truly practical.

[asdf88990]

If you look at energy consumption per capita and adjust for global production, you will see that the Chinese are almost at the very top.

It is of course given that in raw numbers the kitchen and biller-room will consume more energy in the household, but looking at raw numbers is shallow.

[pixl97]

China does have its own set of cares, they may be different than ours but they still exist. If some open Chinese model goes nuts and posts Winnie the Pooh memes everywhere in China you should expect said models to get yanked off the market, and said creators might end up with a rope around their neck.

[matheusmoreira]

Low risk. The western AI models censor even more wrongthink than the chinese ones, not even kidding. Besides, once we have the weights, we can just undo the censorship.

[deadbolt]

> While the west worries about climate change, China burns more coal than ever before.

I'd wager the majority of the visitors of this site are smart enough to not fall for this. What are you doing?

[pc86]

What is it you're accusing them of?

[Chance-Device]

It’s not fear mongering though, is it? These models do have the cyber offensive capabilities claimed. Could Mythos walk someone through gain of function experiments on some virus? I’m pretty sure it could. We’re more protected by limited access to lab equipment and reagents than by difficulty.

The sad truth is that a lot of people are not going to believe it until something happens and people die. Successfully preventing that from happening will be seen as evidence that the prevention wasn’t needed.

[AnthonyMouse]

> These models do have the cyber offensive capabilities claimed.

People use this argument against every new technology. We need to license these new printing presses or subversive elements will use them to publish seditious literature. We need to ban strong encryption or the government won't have invisible warrantless access to everyone's private messages, think of the children. 3D printers can be used to make gun parts -- as can a variety of ordinary tools people commonly have at home, but never mind that bit.

> The sad truth is that a lot of people are not going to believe it until something happens and people die. Successfully preventing that from happening will be seen as evidence that the prevention wasn’t needed.

A 12 oz bottle of water is too dangerous a technology for ordinary people to have on an airplane. Four 3 oz bottles and an empty 12 oz bottle to pour them into after passing through security is totally fine though, naturally. And we need to keep this up forever, or don't you remember 9/11?

The issue here is not that it's impossible for 12 oz of unknown liquid to damage an airplane.

[matheusmoreira]

> These models do have the cyber offensive capabilities claimed.

So? That's like saying "these guns do have the bullet shooting capabilities claimed".

I want all of those cyberwarfare capabilities for myself, precisely so I can defend myself from the onslaught that's coming whether they regulate it or not. This "lol only a select few ultratrusted gigacorporations get access" thing is absolute nonsense.

It's a front for regulatory capture, it's the means for pulling up the latter behind them, for ushering in the technofeudalism that will put us all in the permanent underclass. I simply refuse to accept any of it. If people die that's the price of freedom.

> We’re more protected by limited access to lab equipment and reagents than by difficulty.

As it should be.

[mschuster91]

> I'm really looking forward to the day the chinese finally start manufacturing memory and GPUs. We desperately need more competition in this area to collapse hardware prices and make local AI models viable.

Yeah and if the quality of that memory is like Chinese steel (which is called "chinesium" for a reason), eventually all we'll get is enshittification. Premium binned memory or ECC is for the rich and the rich only, and the rest of us has to pray their memory won't bitflip while something important is stored there.

[CamperBob2]

If you want the cheapest shit grade of steel, they will sell it to you. If you want the best grade available anywhere, they will sell that to you as well.

It's not a matter of the Chinese being incompetent, it's a matter of the buyer demanding the lowest price possible and/or not paying attention to what they receive.

[linzhangrun]

China has already been manufacturing memory and AI GPUs for a long time.

CXMT is now the world's fourth-largest DRAM manufacturer, with about 7.7% market share in 2025; YMTC has about 13% of the global NAND market.

Meituan's newly released 1.6T LongCat was trained entirely on Huawei cards. DeepSeek, Qwen, GLM and others are also actively doing domestic-card adaptation and replacement.

[matheusmoreira]

Better than being straight up priced out of computing altogether I guess.

[mensetmanusman]

They won’t when citizens run AI on their phone that contradicts Xi thought

[lovich]

Do the Chinese models have anything to say about Tiananmen Square? Or if they can act as a surrogate girlfriend/boyfriend?

Both countries are engaging in different flavors of censoring.

[matheusmoreira]

Once we've got the weights, anything is possible.

https://github.com/p-e-w/heretic

[lovich]

How does this work? I don’t have a setup to evaluate it atm.

[bg24]

Follow the money, eg. investors and their connections to the Govt and media.

[9dev]

It's such a freak incident of history that right at this critical time the dumbest, most incompetent leadership is at the helm in the US...

[XorNot]

This is marketing.

Frankly I'm inclined to say that it might also be faked: this drops just days after a new Chinese model does with the usual effect on OAIs projected stock price?

[Chance-Device]

It’s marketing the same way shitting your pants in public is marketing. People notice you.

[rocqua]

It’s more like eating your own fiber supplement in public, and then shitting your pants and telling everyone about it. Sure, it’s embarrassing. But it shows how potent your product is.

[krick]

Apparently this is totally legit marketing strategy now. It truly is, especially if there are enough people who think that shitting your pants is cool, and the people that form the "market" nowadays may have a very different idea from yours about what is cool. Their ideas about coolness are very different from mine, that's for sure.

[asdf88990]

Obviously shitting your pants in public shows you have a healthy digestive system and if you can demonstrate byproducts of wild food in your output, you’re approaching independent thinking and self-reliance.

This is how the financiers look at this and whatever you think it is right or wrong, it does showcase “capability”.

[duzer65657]

Remember when the ebola-infected monkey escaping containment was our worst possible nightmare? Now it's apprently some sort of tech-bro flex to be celebrated.

[J_Shelby_J]

Anyone can have bad security. No one cares. But you can convince those who don’t know better that breaking bad security with an LLM is a once in a civilization investing opportunity. You just need to convince a handful of billionaires and market makers to get on board.

How much would someone have to pay you to take the fall for bad security? A million? A billion? 500b? The stake at play puts it in the realm of geopolitics.

[idiotsecant]

I think all of western (or at least American) discourse of all kinds has recently devolved into who can shit their pants the loudest. I'm hardly surprised when it becomes a dominant advertising strategy

[0xDEAFBEAD]

You guys have created this un-falsifiable "marketing" narrative. Why is it that Jensen is pushing back on the doomer stuff, and complaining that it is hurting AI investments?

https://www.businessinsider.com/nvidia-jensen-huang-ai-doome...

[AlexandrB]

Jensen Huang wants to sell more hardware, he doesn't care whether it's Anthropic, OpenAI, or some Chinese company that wins. So he has a vested interest in AI being perceived positively so that the construction of datacenter continues unopposed.

OpenAI and Anthropic have completely different motives, they're in a zero-sum game with each other and with cheap Chinese models. Regulatory capture that results in artificial barriers of entry is their best bet at healthy profit margins even if it comes at the cost of less overall AI buildout.

It's not like "the AI industry" is a single entity with a single purpose, these are different companies with different objectives.

Edit: Google pretty much spells out the problem that AI labs are faced with in the leaked "We have no moat" memo[1]:

> People will not pay for a restricted model when free, unrestricted alternatives are comparable in quality. We should consider where our value add really is.

And

> All this talk of open source can feel unfair given OpenAI’s current closed policy. Why do we have to share, if they won’t? But the fact of the matter is, we are already sharing everything with them in the form of the steady flow of poached senior researchers. Until we stem that tide, secrecy is a moot point.

> And in the end, OpenAI doesn’t matter. They are making the same mistakes we are in their posture relative to open source, and their ability to maintain an edge is necessarily in question. Open source alternatives can and will eventually eclipse them unless they change their stance. In this respect, at least, we can make the first move.

[1] https://newsletter.semianalysis.com/p/google-we-have-no-moat...

[0xDEAFBEAD]

But this incident undermines OpenAI's safety case compared with open models.

Furthermore how do you explain Sam's downplaying? https://xcancel.com/HumanHarlan/status/1965932275465597077#m

[ycsux]

This is marketing, totally. HF conveniently created a weak sandbox

[lenerdenator]

> There’s been a relatively big reaction to Kimi K3 and Chinese open weights models, but only for financial reasons.

Let's be honest: it's financial and national security reasons.

China has a long and storied history of hacking attacks on American and western targets.

There are other parts of the world that make open weight models; Mistral is a European option. You don't see the worry about that because most people in the US are used to existing in a world order where European powers are considered ambivalent to the US at worst and holders of a special political relationship at best.

If Mistral had the same backing that Chinese AI companies did, there probably wouldn't be as much hemming and hawing. Sure, American companies would take a haircut, but that haircut wouldn't be seen as a move towards software hegemony built on top of manufacturing hegemony. It'd just be you calling into Paris or Frankfurt to talk to your vendor in the future.