[villish]

> Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.

There will be a generational leap that makes current LLMs obsolete in every way. Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.

Humans for millennia have used technological advances to make better weapons. AI will be no different, unfortunately. Wars will be fought with drones and artificial intelligence from now on.

[marcus_holmes]

> Let's say China gets there first, why would the CCP let it be released publicly? That is an insanely valuable advantage in everything from war fighting to economics and more.

Because China is based in a completely different set of values than the USA. They look at the world differently, and have a different understanding of it.

Your assumption that your attitude is pure human nature is deeply grounded in your culture, which (assuming you are from the USA) is deeply grounded in personal competition rather than co-operation. All that rugged individualism. Other cultures are less based around that, and don't see the world as a zero-sum competition for survival.

I'm no expert in Chinese culture, but there are folks from China on this topic who could probably answer this from their perspective.

[Gormo]

Culture does not exist in a vacuum -- it's downstream of fundamental human motivations that are manifest in all times and places.

And using "culture" as a general concept to dismiss concerns about specific behavior, without actually attempting to posit any particular cultural values, or describe what mechanism actually causes those values to have a determinant effect on behavior, is equivalent to saying nothing at all.

Whatever official party line the current Chinese government espouses, it's plenty clear from even a cursory understanding of Chinese history, from the Warring States period to the warlord-era conflicts of the early 20th century, to the current behavior of the CCP, that China does not have any effective cultural guardrail against factions seeking advantage over opposing factions, which is a phenomenon that has been present in all societies that have ever existed in human history.

[fc417fc802]

This doesn't pass the sniff test.

Consider that the US was intimately involved with the start of the FOSS movement. Compilers and commercial operating systems are hugely valuable.

Also consider that history the world over is littered with counterexamples. Historical China itself is rife with examples of technological secrecy and exclusivity.

No competent government will voluntarily release (or allow to be released by its subjects) technology that would give them a meaningful advantage on the world stage.

[marcus_holmes]

The recent Stratechery piece talks about some of this, most relatably "commercialise your competitors" [0].

By releasing their models, they effectively sabotage the entire US AI industry, and given that that is underpinning a sizeable % of US GDP at the moment, therefore sabotage the US economy.

That may not be their intention. The USA is also their biggest customer, after all. But it does show that there are other strategies that might occur to China that don't reflect the knee-jerk reflex to restriction and control that Anglosphere governments have around tech.

[0] https://news.ycombinator.com/item?id=48977128

[fc417fc802]

> it does show that there are other strategies that might occur to China that don't reflect the knee-jerk reflex to restriction and control that Anglosphere governments have around tech.

I'm not sure why you think that is necessarily reflexive nor why you think China differs. The current cost benefit is simply different for them than it is for the US. I expect the US might do the same if the tables were turned.

Disregarding historical examples, this is the country that implemented the "great firewall". That alone leaves me perplexed at your position here. Clearly they are happy to restrict and control when it suits them.

[breezybottom]

"I'm no expert in Chinese culture"

You've made that plenty clear.

[marcus_holmes]

What a bizarre comment to make.

[owebmaster]

Are you an expert?

[somenameforme]

You're making an assumption that's debatable, but let's just assume it's correct. The reality is that China won't be the only to achieve such. In the same way that the US put arguably the best minds in the world towards developing the atom bomb, and within a few years the Soviets had theirs as well. Shortly thereafter they were popping up all over Europe. It's kind of the story of LLMs in general - once people saw it was genuinely possible, we went from one LLM company to them showing up everywhere.

It's highly improbable that any country retains a meaningfully persistent technological lead, and the soft power to be gained from providing models cheaply for society will probably greatly outweigh any sort of temporary strategic advantage that 'hoarding' could provide. And the reality is that nuclear weapons remain the kings of war, by an overwhelmingly wide margin. If the US and China or Russia got into a "real" war, that's pretty much the end of the world.

[villish]

> You're making an assumption that's debatable, but let's just assume it's correct. The reality is that China won't be the only to achieve such. In the same way that the US put arguably the best minds in the world towards developing the atom bomb, and within a few years the Soviets had theirs as well.

I never said other countries won't gain access to the technology. The point is that the 1st country to achieve that leap will get to use it to gain advantages in literally every sector.

I get some people are hoping that AI research is for the benefit of all, and I do too, but that is wishful thinking.

[bigyabai]

> The point is that the 1st country to achieve that leap will get to use it to gain advantages in literally every sector.

That's still speculative. We have something of a basis for speculation with the way LLMs influenced software engineering, but it's not necessarily applicable to "literally every sector" of scientific progress. If their quantum leap is anything like vibe coding, then it will quickly become an unmanageable cesspit lacking accountability and grasping at notoriety. We have precious little to show for half a decade of LLM-generated code, nobody disrupted the FAANG incumbents by using an LLM to crack the digital secrets of the universe.

The safety angle played its card and lost. We've been generating trillions upon trillions of tokens, and nobody has raised the dead or made The Bomb with red mercury. Smarter AI clearly isn't a panacea.

> I get some people are hoping that AI research is for the benefit of all, and I do too, but that is wishful thinking.

You're way too lost in the plot. Technology can absolutely be used for ill, and no doubt AI has already contributed to reprehensible things. But it cannot turn lead into gold. It cannot reveal the face of God or compute The Answer to the Ultimate Question of Life, the Universe and Everything. The ultimate threat of AI is the stuff it can already do - cosine similarity search on billions of discrete records, automated labeling with computer vision, multimodal analysis and corroboration pipelines. The hypothetical scenario where superintelligence is made is a thought-terminating cliche that is driving people into a zealous fervor.

[villish]

Yes I am speculating a leap past LLMs. They are on the edge of what I would actually consider AI.

I believe it is an inevitability and I'm not speculating whether it's in 10 or 50 years.

[bigyabai]

Can you substantiate your belief, or is it just a feeling?

I'm not saying that you're necessarily wrong, but you're certainly lacking evidence. It would be easier to buy into your speculation if you could explain what limits current LLMs, and how they could be optimized to create a step-change in prose generation or reasoning capability.